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Abstract 

We propose an improved LASSO estimation technique based on Stein-rule. We shrink 
classical LASSO estimator using preliminary test, shrinkage, and positive-rule shrinkage 
principle. Simulation results have been carried out for various configurations of correlation 
coefficients (r), size of the parameter vector (/3), error variance (cr^) and number of non-zero 
coefficients (fc) in the model parameter vector. Several real data examples have been used 
to demonstrate the practical usefulness of the proposed estimators. Our study shows that 
the risk ordering given by LSE > LASSO > Stein-type LASSO > Stein-type positive rule 
LASSO, remains the same uniformly in the divergence parameter as in the traditional 
case. 

Keywords and phrases: LASSO; Shrinkage-LASSO; Preliminary test LASSO estimator; 
Positive rule LASSO estimator; 


1 Introduction 


The estimation of par ameters o f a mo del with “uncertain prior information” on parameters 


of interest began with 


BancroftI (j 19441 ) in the classical front. But a breakthrough came when 


1 





SteinI (1956|) and 


James and SteinI (jl961l ) proved that the sample mean in a multivariate 


normal model is not admissible under a quadratic loss, for dimension more than two. This 
very result gave birth to a class of shrinkage estimators of various fo rm and setup. A partia l 


document on preliminary test and Stein-type estimators are given bv l Judge and BockI (jl978l ). 


The Stein-type estimators have been reformulated and expanded by 


Salehl (l20na Ch 4.4.3) 


which includes asymptotic and nonparametric methods. Due to the immense impact of 
Stein’s results, scores of technical papers appeared in the literature covering various areas 
of applications. Here is one with the popular LASSO estimator. 

This paper is devoted to the study of the performance characteristics of several improved 
estimators of LASSO based on preliminary test and Stein’s principle, and the comparison 
of the LASSO estimator with the least square estimator (LSE), improved preliminary test 
estimator (IPT), shrinkage preliminary test estimator (SPTE), Stein-type shrinkage LASSO 
estimator (SLE), and Stein-type positive-rule shrinkage LASSO estimator (PSLE). An im¬ 
portant characteristic of LASSO-type estimators is that they provide simultaneous estima¬ 
tion and selection of coefficients in linear models and can be applied when dimension of 
the parameter space exceeds the dimension of the sample space. Our conclusions are made 
based on simulated mean-squared errors (MSE) and relative efficiency tables and graphs. 
It is shown that the modified LASSO carries on with the same dominance characteristics 


as the usual preliminary test and Stein-type estimators (| Salehl. 


20061 ). 


Our contribution in this paper is unique. We have proposed a set of LASSO-based 
shrinkage estimators that perform superior to the classical LASSO estimator. We studied 
the theoretical properties of the estimators in terms of asymptotic mean squared errors 
(AMSE). Analytical expressions for the asymptotic risk functions of the proposed estima¬ 
tors have been provided. We carried out Monte Carlo simulation experiments to study the 
risk-behavior of the proposed estimators and their comparisons with the LASSO estima¬ 
tor. Application of the proposed estimators have been demonstrated using three real life 
data examples where the proposed estimators performed superiorly to the classical LASSO 
estimator. 

The organization of the paper is as follows. Section 2 contains the basic informations 
about the LASSO and preliminary test and Stein-type estimators. Proposed improved 
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estimators of LASSO are presented here. Risk properties and risk-comparisons of various 
estimators are presented in this section. In section 3 we outline our simulation setup and 
discuss simulation results. Details of the analysis of relative efficiencies of estimators are 
presented in this section. In section 4, applications of the proposed estimators have been 
demonstrated using three real life data sets. Finally, conclusions are provided in section 5. 

2 Linear model and estimators 

Consider the linear multiple regression model 

Y = Xp + e ( 2 . 1 ) 

where Y = {yi,y 2 , ■ ■ ■ ^yn)'■, P = {/3i,/32, ■ ■ ■ ,f3py, X is the n x p design matrix and e = 
(ei, 62 ,..., e„)' iid error n-vector. Further we assume that E{e) = 0 and E{e'e) = cr'^In- 
It is well-known that the LSE of f3 is given by 

Pn = {X'X)-^X'Y (2.2) 

which we use for obtaining preliminary test and shrinkage estimators. LSE f3n is the “best 
linear unbiased estimator (BLUE)” of /3. The solution (12.21) depends on the non-singularity 
of the matrix X'X. If it is singular, the uniqueness of fin is lost and we end up with 
multiple solutions with varying variances and some of them may be very large. To avoid 
these problems, a class of penalty estimators evolved in the class of restricted estimators. 
A simple example of “restricted estimator” when we want an estimator of /3 which belongs 
to the subspace defined by iL/3 = /i, where H is a q x p matrix and h is a g-vector or real 
numbers. 

Next, we consider the classical penalty estimator, called the LASSO (least absolute 
shrinkage and selection operator) belonging to a class of restricted estimators. 

The estimators of /? is obtained by minimizing the LS criterion subject to Hf = h. 
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Explicitly we may write 


mm (y - XP)'{Y - Xj3) + \{HI3 - h). 


The solution for this problem is the “restricted estimator” and the tuning parameter A can 
be explicitly obtained giving 


/3n = l3n- -h), C = X'X, 


(2.3) 


where /3„ = (X'X)-^X'Y, the LSE. 

For this, we consider \ f^j\ < ^ as the restriction and minimize 


mm (F - XPYiY - X/3) + A \/3j 

i=i 


yielding the solution as LASSO introduced bv lTibshiranil (jl996li . and is given by 


(2.4) 


Pn = 


Ya=i (X)j=i XijXikPj - XikVi^ + |sgn(/3fc) = 0 
and /3k = 0 


(2.5) 


When ^X'X —>■ Ip, the solution may be written as 


/3n = 


n on 

In’ * * * ’ r'pn 


= Sgn0jn) ( 0jn\ “ 77 ) J = 3,---,P 


( 2 . 6 ) 


where = (|/3i„,|, ..., |/j„„,l) ■ 


Actually 


Frank and FriedmanI ()1993l ) defined the class of generalized version of LASSO, 


namely, the “bridge estimator” as 


min (y - X0)'(Y - X0) + A ^ IftP- 


(2.7) 


If 7 = 1 the solution reduces to the LASSO estimator. 
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LASSO proposed by 


Tibshiranil (jl996r ) simultaneously estimates and makes selection of 


variables with appropriate interpretation and its vira 
putational solutio n and methodology see 


Tibshirani 


Efron et al 


and 


popu larit y in applications. F or com- 


19961 1 and 


Efron et al 


( 2004 1. Later 


(I 2 OO 4 II p r opose d Least Angle Regression (LAR) which is a stepwise regression, 


Friedman et al 


([20101) developed an efficient algorithm for the estimatio n of a GLM 


with c onvex penalty. During the course of development of penalty estimators, 


Fan and Li 


(|200l[ l defined good penalty functions as the one which yield (i) nearly unbiased estimator 


when true parameter is large to avoid unnecessary modeling bias, (ii) an estimator which is 
a threshold rule that sets small estimated coefficients to zero to reduce model complexity, 
and (iii) the resulting estimator to be continuous in the data to avoid instability in the 
model prediction. In this paper, we present an improved version of LASSO. 


2.1 PTE and Stein-type estimators 

For the linear multiple regression model, Y = A/3 + e, if we suspect the full hypothesis to 
be /3 = 0 (null-vector), then the restricted estimator (RE) j3n = 0 and the test for /3 = 0, 
vs /3 / 0 may be based on the statistic 


C 


n — 



( 2 . 8 ) 


where 

sl = {n-p)-\Y-XPnnY-X^n)- (2.9) 


Under the conditions 

(1) ^A'A —>■ C as n —>■ 00 (C positive definite ), and 

(2) msiXi<j<nXj{^X'X)~^Xj —0 as n ^ 00 

where Xj is the jth row of A, Cn —>■ Xp -central chi-square variable with p degrees of 
freedom (df). 
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Let Xp(ct) be an upper a-level critical value from this null distribution; then we may 
define the preliminary test estimator (PTE) of (3 as 

= Pn- Pnlix^ <xlia))- (2-10) 

The PTE is a discrete variable. As a result some optimality properties when we consider 
assessing its MSE comparison is lost. We may define a continuous version of PTE as the 
James-Stein-type estimator (JSE) given by 

P^ = Pn-{p-2)PnC-\ ( 2 . 11 ) 

Note that we have replaced I{x^ < Xp(®)) by {p — in the definition of PTE. However, 

has an inherent problem of changing its sign due to the factor (1 — (p — 2)C~^) which 
may be larger than 1 in absolute value. If that happens, from applied point of view, its 
interpretation becomes blurred. Thus, we define another estimator, namely, the positive- 
rule Stein-type estimator (PRSE) as 

Pt = PnIi^n>{p-2)). ( 2 . 12 ) 


Next, we define an improved preliminary test (IPT) estimator defined by 

pr = /3r (1 - (P - 2)/:-^) . (2.13) 


Thus, to set the stage, we have defined six estimators, namely, LSE, RE, PTE, JSE, PRSE, 
and IPT here, and one penalty estimator LASSO. Next, we use the definitions above to 
propose new shrinkage-type LASSO estimators. 


2.2 Proposed improved estimators of LASSO 


Let us redefine the LASSO estimator (LE) (|Tibshiranil . 


1996 1 as 


Pn^ = argmin {y - XI3)'{y - XP) + A V |/3j 
/3 ^ 


(2.14) 


i=i 
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From now on, we consider as our unrestricted estimator (UE). Then, similar to the 
definition of PTE, we define the preliminary test LASSO estimator (PTLE) as 


^ptle ^ pLE 0 < a < 1 (2.15) 

The Stein-type shrinkage LASSO estimator (SLE) based on (3^^ may be defined as 

= (2.16) 

where Cn was defined in (12.Sp . Now we define the Stein-type positive rule shrinkage LASSO 
estimator (PSLE) using and Cn as 

I{C-n >{p- 2)). (2.17) 


We note that Cn follows a non-central distribution with noncentrality parameter = 
^ under local alternatives. 

2.3 Orthogonal case, r = 0 

In this section, we consider the model (j2.ip and assume that the design matrix is centered 
and ^(X'X) = Ip. Under this condition, the LASSO estimator is given by 


/3^''(A) = (/3i“ ...,/3“)', 


(2.18) 


where = sgn {^i n){\l3.in\ - A)J(|/3j„ | ^x) for j = 1,... ,p. 


It is known from 
given by 


Donoho and Johnstonel (|l994ll as n ^ oo, the quadratic risk bound is 


m= ^(l + 2np)(l + A^.^) 


P - 

where 1) under the local alternative 

Pjn = <5 = ((5i,..., 5p)' / 0 for A = a\j2np. 
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For the sparse solution, one has to use Amin = k as such m equals m* = ^(1 + 2lnp)(l + k) 
since we have k{< p) coefficients satisfy Sj > a'^ and remaining equal to zero. 

Now, we consider the PTE, SE and PRSE as defined in section (2.1) where we take 
£^n = , Thus, one can hnd that the asymptotic risk-bound of the PTE, SE and PRSE 

are then given by 

2 

= —(1 + 21np)(l + fe) = u‘^m* 

P 

s 

R20n) = R20n^) = —T = Note that restricted estimator is zero, 
cr^ 

K(1 - ?^fc+2(c«; A2)) + a 2 {21Rfc+2(ca; A^) - Uk+Aicc.-, A^)}] 

K - (p - {2 E[x^2^(A2)] -{k- 2)E[x^4^(A2)]} 

+(p2-4)A2E[x^^4(A2)]] 

RSu^^^) = R^n^^) - c7^m*E[{l -{k- 2)xlUl^^)?I{xl+2{^^) <k-2)] 

+ u^A^ {2E[(1 -{k- 2)x-kU^^)fl{xl+,{^^) <k-2)] 

-E[{1 -{k- 2)xlU^^)fl{xl+A{^^) <k-2)] 

Here, 7^p+2i/(ca; A^) is the cdf of a noncentral chi-square distribution with p + 2u di and 
noncentrality parameter A^, and 

POO 

E[x-lU^^)] = x-^^dnp+2uix;A^). 

2.4 Analysis of asymptotic MSE of the estimators 

Eirst, we note that Ri(/3„) > uniformly in A^. Next, we compare and 

by taking the risk-difference 

- R30r^^) = m*nk+2{Ca;A^) - A^ {2nk+2{Ca-, A^) - Uk+AiCa, A^)} I 0 

whenever 

.2 < _ m*Uk+2{Ca, A^) _ 

^ {2'RA:+2(Ca; A^) — l-Lk+AiCa'-i A^)} 
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Note 


'm*'Hk+2 > A^(2^fc+2 — T-ik+i) 

or, < 


2 ^ m*'Hk +2 


2^fc+2 — T-ik+A 


Accordingly, PTLE is better than LE in the range of A^. Otherwise, for 


^2 ^ 'm*V.k+2 


‘^T-Lk+2 — T-Lk+A 


LE is better than PTLE. Next, we note that for all A^, the risk-differences 


(i) Ri0n^) - > 0 for all A 


SLE\ 


and risk-difference 


[ii) RA0n^^) - R^iPnPSLE) > 0 for all A"" € (0, oo). 


Hence, 


< RAiP^^^) < RM^) VAT 


dSLE 


LE\ wa2 


3 Simulation for orthogonal case 

For the non-orthogonal case, we conduct Monte Carlo simulation experiments to study 
the performance of the proposed positive-rule shrinkage-LASSO estimator (PSLE) along 
with preliminary test LASSO estimator (PTLE), and shrinkage LASSO estimator (SLE). 
In particular, we study r elative efficiencie s of the proposed estimators compared to the 


LASSO estimator (LE) bv lTibshiranil (|l99f)l ). In the simulation studies, mean squared errors 


(MSE) were computed for each of the proposed estimators, and their relative efficiencies 
were calculated by t aking the ratio of MSE of the proposed estimators to the MSE of LE. 


Raheem et al 


(j20I211 have conducted similar studies where a sub-hypothesis was tested and 


relative efficiencies of various shrinkage and penalty estimators were studied in a partially 
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linear regression setup. In this stud y, we are concerned with fu ll model hypothesis Hq : /3 = 0 


against the alternative Ha : P 0. 


Saleh and Raheeml (j2015l ) have studied the performance 


of various shrinkage and penalty estimators under a full model hypothesis using the same 
setup. Next, we discuss the simulation setup. 

First, We generate the design matrix X from a multivariate normal distribution with 
mean vector /x = 0 and covariance matrix 51. The off-diagonal elements of the covariance 
matrix are considered to be equal to r with r = 0,0.5,0.9. We consider sample size to be 
n = 100, and the number of parameters, p, equal to 10, 20, and 50. In our setup, /? is a 
p-vector and a function of A^. In the simulation, the /3 vector is defined such that a = 0 
indicates a data set being generated under null hypothesis, whereas A^ > 0 indicates a data 
set generated under alternative hypothesis. We considered 23 different values for A^, which 
are 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1, 1.5, 2, 3, 5, 10, 15, 20, 25, 30, 35, 40, and 
50. Each realization was repeated 1000 times to obtain bias-squared and variance of the 
estimated regression parameters. Subsequently, MSEs were calculated for the least squared 
estimator (LSE), improved preliminary test estimator (IPT) which is based on LSE, SEE, 
and PSLE (READ/CHECK THESE LINES AGAIN). The responses were simulated from 
the following model: 


Vi = 


xiPi+ Ci 


1=1 

where e 

i iV(0,(T^) with two different values for : 10^,20^. 

Secondly, the data generation setup was further modified to accommodate the number 
of non-zero /3s in the model. In particular, we partitioned /3 as /3 = {k, q)' where k indicates 
number of nonzero /3s, and q indicates p — k zeros-a function of A^. To translate the 
above, when p = 10, and k = 3, we would have /3 = (1,1,1,0, 0,0, 0,0,0,0)' to generate 
the response under null hypothesis. When A^ is introduced, e.g., A^ = 5, we would have 
P = (1,1,1,0,0,0,0,0,0, 5)' to generate the response under alternative hypothesis. Clearly, 
inclusion of A^ acts as a degree of violation of the null hypothesis. As A^ increases, so does 
the degree of violation of the null hypothesis. We study the performance of the proposed 
estimators under varying degree of violation of null hypothesis as measured by A^. 
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Finally, the relative efficiencies were calculated using the following formula. 


Relative Efficieicy (RelEff) 


MSE(/3LE) 

MSE(/3*) ’ 


(3.1) 


where is the LASSO estimator, and /3* is one of the estimators whose relative efficiency 
is to be computed. As in equation (13.ip . a relative efficiency greater than 1 would indicate 
superiority of the proposed estimator compared to the LASSO estimator. On the other 
hand, a relative efficiency of equal to or less than 1 would indicate that the efficiency of the 
estimator i s at or below that of the LASSO estimator. We used R statistical software (' R 


Core Team, 


2014li to carry out t 


R package (jEriedman et al. 


le simulation. Eor obtaining LASSO estimate, glmnetO 


20141 ) was used. 


In the following, we discuss the results of our simulation studies. 


3.1 Discussion of simulation results 

In this study, data have been generated with correlation between the x’s, r = 0,0.5,0.9 
for n = 100, p = 10,20,50, and the error variance = 10^,20^. The relative efficien¬ 
cies of the proposed estimators are presented in Tables [7] through [231 To visually compare 
the results of various configurations, relative efficiency of the estimators are compared to 
LASSO estimator as shown in Eigures flOl through IT^ Since positive-rule shrinkage-LASSO 
estimator (PSLE) outperforms all other estimators for most of the configurations, we sep¬ 
arately compared its performance for p = 50 and cr = 20 at various correlation coefficient 
r = 0.0,0.5,0.9, and the results are displayed in Eigures [7] through [H In all of these fig¬ 
ures, a horizontal line was drawn at 1 on the y-axis to facilitate the comparison among the 
estimators. Eor a given estimator, any point above this line indicates superiority of the 
estimator compared to the LASSO estimator in terms of relative efficiency. 

The findings of simulation studies may be summarized as follows. 

(i) The performance of LSE, LE, SLE and PSLE may be ordered as PSLE > SLE > LE 
> LSE (where > indicates dominance) for all A^. See Tables (711241 

(ii) LASSO dominates over LSE uniformly. 
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(iii) At any given p and relative efficiency of the estimators is a decreasing function of 

both k and A^. As and k increases, the relative efficiency decrease. 

(iv) Gain in relative efficiency of the estimators is a decreasing function of the number 
of /3s that are zero as indicated by k. Figures [101 EH and [l2] show that the relative 
efficiency of SLE and PSLE are the highest when k = 0 (where relative efficiency is 
above 9). Eor k = 1 the relative efficiency is around 6. The relative efficiency is around 
4 for k = 3 and 5. 

(v) PTLE dominates IPT uniformly. However, IPT dominates the least squares estimator, 
which is consistent with the results found in literature. 

(vi) Neither PTLE nor Stein-type LASSO estimator dominate each other. 

4 Real data examples 

In the following, we study three real life examples. We pre-process the data sets by centering 
the predictor variables. We then fit linear regression models to predict the variable of 
interest using the available regressors. Lasso estimator (LE), improved preliminary test 
(IPT), Stein-type shrinkage LASSO (SLE), and Stein-type positive-rule shrinkage LASSO 
(PSLE) estimators are then obtained for the regression parameters. 

We evaluate the performance of the estimators by computing average cross validation 
error using K = lO-fold cross validation. In cross validation, the data set is randomly 
divided into K subsets of roughly equal size. One subset is left aside, termed as test set, 
while the remaining K — 1 subsets, called training set, are used to fit the model. The fitted 
model is then used to predict the responses for the test data set. Finally, prediction errors 
are obtained by taking the squared deviation of the observed and predicted values in the 
test set. 

In cross validation, the estimated prediction error varies across runs. Therefore, we 
repeat the process 1000 times, and calculate the average and standard deviation of the 
prediction errors. We found 1000 to be large enough number of runs to stabilize the standard 
deviations as no noticeable changes were observed for larger values. 
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4.1 Galapagos data 


Faraway (j2002ll analyzed the data about species diversity on the Galapagos islands. The 


Galapagos data contains 30 rows and seven variables. Each row represents an island, and 
the covariates represent various geographic measurements. The covariates are: the number 
of endemic species, the area of the island (km^), the highest elevation of the island (m), the 
distance from the nearest island (km), the distance from Santa Cruz island (km), the area 
of the adjacent island (km^). The original data set contai ned missing values for some of 


the covariates, which have been imputed by 


Faraway! (|2002f ) for convenience. The response 


variable is the number of species of tortoise found on the island. 

The summary statistics in shown in Table [TJ The visual correlation matrix for the 
centered covariates is shown in Figured) 


Table 1: Summary statistics for the Galapagos Data. 


Min 

Q1 

Median 

Mean 

Q3 

Max 

SD (Response) 

2.00 

13.00 

42.00 

85.23 

96.00 

444.00 

114.63 


Table 2: Average prediction errors and standard deviations for the estimators based on 
K = 10-fold cross validation, repeated 1000 times for Galapagos data. 


Estimator 

Bias Corrected CVE 

Average 

SD 

EE 

76350.90 

15057.76 

PTEE 

76350.90 

15057.76 

SEE 

70569.66 

13152.35 

PSEE 

70569.66 

13152.35 


Figure [2] displays the prediction error of the estimators for 1000 cross-validated runs. 
The average and standard deviation of the predictor errors are summarized in Table [2l It 
is noted here that the prediction errors are unusually large for this data set. The reason is 
due to the variability present in the original data set (standard deviation for the response 
variable is 114.63). 

We hnd SEE and PSLE to be performing better than the LE and PTLE. Notably, SEE 
and PSEE have smaller standard deviations compared to EE and PTEE. 
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Figure 1: Correlation matrix for Galapagos data. 
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Figure 2: Prediction error for the proposed estimators for Galapagos data. 
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4.2 State data 


Faraway (j2002f ) illustrated variable selection methods using the state data set. There are 


97 observations (cases) on 9 variables. The variables are: population estimate as of July 1, 
1975; per capita income (1974); illiteracy (1970, percent of population); life expectancy in 
years (1969-71); murder and non-negligent manslaughter rate per 100,000 population (1976); 
percent high-school graduates (1970); mean number of days with minimum temperature 32 
degrees (1931-1960) in capital or large city; and land area in square miles. We consider life 
expectancy as the response. 

Summary statistics for this data set is given in Table El Correlation coefficients between 
the predictors is displayed in Figure El We notice moderate to strong correlation present 
between some of the predictors. In the simulation studies, we have observed that the 
proposed estimators perform superiorly when the correlation between the predictors is large. 


Table 3: Summary statistics for the State Data. 


Min 

Q1 

Median 

Mean 

Q3 

Max 

SD (Response) 

67.96 

70.12 

70.68 

70.88 

71.89 

73.60 

1.34 


Table 4: Average prediction errors and standard deviations for the estimators based on 
K = 10-fold cross validation, repeated 1000 times for State data. 


Estimator 

Bias Corrected CVE 

Average 

SD 

LE 

5786.94 

148.94 

PTLE 

5530.89 

50.70 

SLE 

5216.48 

195.24 

PSLE 

5552.93 

81.89 


Table m gives averages and standard deviations of the predictor errors for the estimators. 
For this data, SLE has the smallest average prediction error followed by PSLE. LE has 
the largest average prediction error. Eigure 01 shows the average prediction errors for the 
estimators visually, which demonstrates smaller yet highly variable prediction errors for the 
SLE. 
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Figure 3: Correlation matrix for State data. 
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Figure 4: Prediction error of the proposed estimators for State data. 
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4.3 Longley data 


Longley data set (jLongleyl, 


19671 ) is a popular macroeconomic data set and is widely used 


to demonstrate the application of collinear regression. The data has 16 observations on 
seven variables. The predictors are GNP implicit price deflator, gross national product, the 
number of people unemployed, number of people in the armed forces, noninstitutionalized 
population aged 14 and older, the year (time). The response variable is the number of people 
employed. Table [5] shows data summary while visual correlation matrix for the predictors 
is shown in Figure [5j Note that the predictors are highly correlated. Table [6] shows SLE to 
be the best estimator in terms of prediction errors followed by PTLE and PSLE. LE has 
the largest average prediction error. Although SLE has the smallest prediction error, it has 
the largest variability. Of the four estimators, PTLE has the smallest variability. The plot 
of 1000 prediction errors in Eigure El demonstrates SLE as the best performing estimator. 


Table 5: Summary statistics for the longley Data. 


Min 

Q1 

Median 

Mean 

Q3 

Max 

SD (Response) 

60.17 

62.71 

65.50 

65.32 

68.29 

70.55 

3.51 


Table 6: Average prediction errors and standard deviations for the estimators based on 
K = 10-fold cross validation, repeated 1000 times for Longley data. 


Estimator 

Bias Corrected CVE 

Average 

SD 

LE 

5182.43 

99.16 

PTLE 

4751.56 

5.97 

SLE 

3163.05 

337.86 

PSLE 

4759.06 

24.25 
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Figure 5: Correlation matrix for Longley data. 
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Figure 6: Prediction error of the proposed estimators for Longley data. 
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5 Discussion and conclusion 


In this paper, we proposed improved estimation technique for the LASSO estimator based 
on Stein-rule. In particular, we used LASSO estimator to obtain preliminary test LASSO es¬ 
timator (PTLE), Stein-type shrinkage LASSO estimator (SLE), and Stein-type positive-rule 
shrinkage LASSO estimator (PSLE). We studied the performance of the proposed estima¬ 
tors under full-model hypothesis when the parameter space is small relative to sample size. 
Simulation studies have been performed to compare the estimators for various configura¬ 
tions of parameter sizes (p), correlation coefficient between the predictors (r), and the error 
variance (cr^). We used relative-MSE criterion to compare the proposed estimators with 
the classical LASSO estimator. We varied the number of non-zero /3s, and evaluated the 
performance of the estimators under varying degree of model misspecification as guided by 
A^. We have provided relative efficiencies of the proposed estimators compared to classical 
LASSO estimator in Tables [7] through [53] as well as graphically for selected configurations 
as displayed in Eigures 171 through [T2l 

The simulation results demonstrate that the classical LASSO dominates the LSE uni¬ 
formly while PSLE has the smallest MSE among the proposed estimators. In particular, 
PSLE uniformly dominates classical LASSO estimator when the error variance is large 
(cj^ = 20^, in our setup). Also, neither PTLE nor SLE dominates one another. Relative 
efficiency of the proposed estimators increases when there are more near-zero parameters 
present in the model. Performance of the estimators decrease as we deviate from the null 
model. These results are consistent with the properties of traditional preliminary test and 
Stein-type estimators found in the literature. 

We have presented three real life data examples to demonstrate the use of proposed esti¬ 
mators. Average and standard deviations of the prediction errors based on the LE, PTLE, 
SLE, and PSLE have been obtained and compared. The proposed estimators outperform 
LASSO estimator in both average prediction error and standard deviation criteria. While 
PSLE dominated the other proposed estimators in the simulation experiments, the domi¬ 
nance picture was not obvious for the real data examples. As such we conclude that neither 
of the PTLE, SLE or PSLE may outperform each other in all real life applications. We 
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tried some other data sets where the correlation between the predictors are low to moder¬ 
ately strong, and found that LASSO as well as the proposed estimators perform equally in 
those cases. Therefore, we conclude that the improved LASSO estimators would find their 
applications for data sets with moderate to strong correlations among the predictors. 
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Figure 7: Relative efficiencies of LSE, IPT, PTLE, SEE, PSLE, compared to LASSO esti¬ 
mator (LE) when r = 0, p = 50, and a = 20. 
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Figure 8: Relative efficiencies of LSE, IPT, PTLE, SEE, PSLE, compared to LASSO esti¬ 
mator (LE) when r = 0.5, p = 50, and a = 20. 
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Figure 9: Relative efficiencies of LSE, IPT, PTLE, SEE, PSLE, compared to LASSO esti¬ 
mator (LE) when r = 0.9, p = 50, and a = 20. 
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Figure 10: Relative efficiency of positive rule shrinkage LASSO estimator (PSLE) compared 
to LASSO estimator (LE) for various k when r = 0. 
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Figure 11: Relative efficiency of positive rule shrinkage LASSO estimator (PSLE) compared 
to LASSO estimator (LE) for various k when r = 0.5. 
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Figure 12; Relative efficiency of positive rule shrinkage LASSO estimator (PSLE) compared 
to LASSO estimator (LE) for various k when r = 0.9. 
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Table 7: Relative efficiencies of IPT, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0, p = 10, u = 10 for 
different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.10 

1.84 

2.50 

4.06 

4.12 

0.17 

1.24 

1.44 

2.08 

2.09 

0.28 

1.24 

1.21 

1.70 

1.70 

0.38 

1.02 

1.03 

1.45 

1.46 

0.1 

0.10 

2.82 

3.04 

3.41 

3.46 

0.16 

1.36 

1.68 

2.10 

2.12 

0.26 

1.06 

1.16 

1.53 

1.53 

0.37 

1.00 

1.04 

1.41 

1.42 

0.2 

0.12 

1.79 

2.37 

3.17 

3.18 

0.17 

1.33 

1.59 

2.04 

2.06 

0.27 

1.07 

1.13 

1.55 

1.55 

0.40 

1.06 

1.04 

1.43 

1.43 

0.3 

0.11 

1.67 

1.62 

2.56 

2.56 

0.18 

1.35 

1.36 

1.97 

1.98 

0.31 

1.10 

1.09 

1.57 

1.58 

0.39 

1.02 

1.03 

1.41 

1.42 

0.4 

0.11 

1.52 

1.78 

2.57 

2.58 

0.19 

1.32 

1.28 

1.88 

1.89 

0.28 

1.07 

1.12 

1.50 

1.50 

0.40 

1.06 

1.04 

1.45 

1.46 

0.5 

0.14 

1.50 

1.53 

2.24 

2.26 

0.18 

1.21 

1.34 

1.78 

1.80 

0.31 

1.10 

1.12 

1.52 

1.53 

0.43 

1.06 

1.03 

1.44 

1.44 

0.6 

0.15 

1.53 

1.61 

2.38 

2.38 

0.21 

1.25 

1.28 

1.87 

1.87 

0.29 

1.07 

1.07 

1.49 

1.49 

0.40 

1.06 

1.07 

1.42 

1.43 

0.7 

0.16 

1.31 

1.45 

2.29 

2.30 

0.19 

1.27 

1.29 

1.77 

1.78 

0.29 

1.01 

1.09 

1.52 

1.52 

0.43 

1.03 

1.01 

1.45 

1.45 

0.8 

0.14 

1.59 

1.82 

2.27 

2.28 

0.22 

1.15 

1.20 

1.76 

1.76 

0.33 

1.05 

1.08 

1.51 

1.51 

0.42 

1.02 

1.03 

1.48 

1.48 

0.9 

0.16 

1.50 

1.44 

2.04 

2.04 

0.20 

1.14 

1.23 

1.66 

1.67 

0.34 

1.06 

1.10 

1.60 

1.60 

0.43 

1.04 

1.01 

1.46 

1.47 

1.0 

0.16 

1.29 

1.37 

2.05 

2.06 

0.20 

1.00 

1.15 

1.68 

1.69 

0.33 

1.05 

1.07 

1.51 

1.52 

0.45 

1.06 

1.05 

1.44 

1.44 

1.5 

0.18 

1.24 

1.27 

1.77 

1.78 

0.24 

1.19 

1.25 

1.71 

1.71 

0.37 

1.08 

1.04 

1.46 

1.46 

0.47 

1.06 

1.03 

1.43 

1.43 

2.0 

0.18 

1.04 

1.38 

1.64 

1.66 

0.25 

1.02 

1.12 

1.56 

1.56 

0.36 

1.01 

1.04 

1.43 

1.43 

0.50 

1.07 

1.00 

1.42 

1.42 

3.0 

0.23 

1.00 

1.14 

1.58 

1.59 

0.28 

0.97 

1.09 

1.51 

1.52 

0.38 

0.93 

0.99 

1.43 

1.43 

0.50 

1.00 

0.99 

1.40 

1.40 

5.0 

0.29 

0.79 

0.91 

1.26 

1.27 

0.32 

0.77 

0.93 

1.28 

1.28 

0.46 

0.88 

0.94 

1.31 

1.32 

0.50 

0.90 

0.93 

1.28 

1.28 

10.0 

0.36 

0.60 

0.78 

1.12 

1.13 

0.34 

0.55 

0.74 

1.04 

1.04 

0.48 

0.74 

0.87 

1.14 

1.14 

0.59 

0.83 

0.88 

1.19 

1.19 

15.0 

0.33 

0.48 

0.71 

0.94 

0.94 

0.39 

0.55 

0.78 

0.98 

0.98 

0.49 

0.67 

0.85 

1.08 

1.08 

0.60 

0.82 

0.91 

1.12 

1.12 

20.0 

0.32 

0.44 

0.79 

0.85 

0.85 

0.40 

0.56 

0.89 

0.94 

0.94 

0.50 

0.69 

0.92 

1.01 

1.01 

0.61 

0.81 

0.96 

1.05 

1.05 

25.0 

0.37 

0.47 

0.84 

0.86 

0.86 

0.39 

0.50 

0.86 

0.87 

0.87 

0.48 

0.63 

0.92 

0.98 

0.98 

0.57 

0.73 

0.96 

1.00 

1.00 

30.0 

0.37 

0.46 

0.90 

0.82 

0.82 

0.39 

0.48 

0.89 

0.82 

0.82 

0.53 

0.65 

0.92 

0.96 

0.96 

0.59 

0.75 

0.96 

0.94 

0.94 

35.0 

0.36 

0.46 

0.94 

0.85 

0.85 

0.41 

0.51 

0.96 

0.85 

0.85 

0.48 

0.61 

0.99 

0.96 

0.96 

0.56 

0.71 

1.00 

0.96 

0.96 

40.0 

0.36 

0.45 

1.00 

0.79 

0.79 

0.41 

0.52 

1.00 

0.87 

0.87 

0.47 

0.57 

0.98 

0.89 

0.89 

0.59 

0.69 

0.95 

0.94 

0.94 

50.0 

0.36 

0.43 

0.97 

0.77 

0.77 

0.36 

0.43 

1.00 

0.81 

0.81 

0.49 

0.58 

1.00 

0.87 

0.87 

0.62 

0.73 

1.00 

0.95 

0.95 



Table 8: Relative efficiencies of IPT, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0, p = 10, u = 20 for 
different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.09 

1.43 

1.89 

3.17 

3.18 

0.11 

1.36 

1.69 

2.66 

2.67 

0.14 

1.39 

1.90 

2.45 

2.46 

0.17 

1.43 

1.40 

1.88 

1.88 

0.1 

0.11 

1.68 

1.80 

3.31 

3.34 

0.13 

1.57 

1.60 

2.68 

2.69 

0.14 

1.32 

1.50 

2.09 

2.09 

0.17 

1.38 

1.45 

2.00 

2.00 

0.2 

0.12 

1.90 

1.95 

3.17 

3.18 

0.11 

1.70 

1.85 

2.62 

2.64 

0.13 

1.41 

1.64 

2.18 

2.19 

0.19 

1.35 

1.32 

1.90 

1.90 

0.3 

0.11 

1.81 

2.17 

3.30 

3.33 

0.12 

1.72 

2.28 

2.75 

2.76 

0.15 

1.26 

1.40 

2.09 

2.10 

0.18 

1.40 

1.39 

1.93 

1.93 

0.4 

0.09 

2.39 

2.50 

3.69 

3.72 

0.14 

1.31 

1.44 

2.52 

2.52 

0.17 

1.35 

1.66 

2.18 

2.19 

0.18 

1.21 

1.31 

1.90 

1.91 

0.5 

0.13 

2.16 

2.23 

3.19 

3.21 

0.13 

1.64 

1.84 

2.75 

2.77 

0.17 

1.50 

1.52 

2.26 

2.26 

0.17 

1.23 

1.43 

1.96 

1.96 

0.6 

0.13 

2.42 

2.22 

3.17 

3.20 

0.10 

1.44 

2.11 

2.61 

2.64 

0.13 

1.32 

1.60 

2.04 

2.04 

0.17 

1.25 

1.22 

1.81 

1.82 

0.7 

0.11 

1.84 

1.80 

3.20 

3.23 

0.14 

1.53 

1.57 

2.36 

2.39 

0.15 

1.55 

1.78 

2.12 

2.14 

0.18 

1.09 

1.26 

1.78 

1.79 

0.8 

0.13 

2.00 

2.35 

3.11 

3.12 

0.14 

1.63 

1.67 

2.53 

2.55 

0.14 

1.42 

1.67 

2.12 

2.13 

0.18 

1.07 

1.26 

1.82 

1.82 

0.9 

0.12 

1.68 

2.05 

3.05 

3.07 

0.13 

1.56 

1.65 

2.55 

2.55 

0.16 

1.31 

1.53 

2.03 

2.04 

0.18 

1.35 

1.37 

1.86 

1.87 

1.0 

0.15 

1.96 

1.82 

2.74 

2.76 

0.14 

1.49 

1.71 

2.38 

2.38 

0.16 

1.40 

1.45 

2.04 

2.06 

0.21 

1.29 

1.28 

1.87 

1.88 

1.5 

0.12 

1.97 

1.64 

2.64 

2.65 

0.14 

1.63 

1.99 

2.44 

2.45 

0.20 

1.31 

1.33 

2.02 

2.02 

0.22 

1.30 

1.27 

1.86 

1.87 

2.0 

0.12 

1.88 

2.07 

2.52 

2.53 

0.12 

1.27 

1.59 

2.16 

2.17 

0.15 

1.28 

1.40 

1.80 

1.81 

0.19 

1.21 

1.29 

1.75 

1.76 

3.0 

0.16 

1.42 

1.50 

2.18 

2.18 

0.16 

1.30 

1.41 

2.06 

2.07 

0.18 

1.28 

1.39 

1.80 

1.81 

0.19 

1.21 

1.34 

1.69 

1.70 

5.0 

0.17 

1.37 

1.39 

1.87 

1.88 

0.18 

1.08 

1.26 

1.83 

1.84 

0.20 

1.17 

1.34 

1.72 

1.73 

0.25 

1.18 

1.17 

1.62 

1.63 

10.0 

0.23 

1.04 

1.16 

1.61 

1.62 

0.26 

0.96 

1.12 

1.60 

1.61 

0.27 

1.11 

1.13 

1.55 

1.55 

0.32 

1.06 

1.03 

1.53 

1.53 

15.0 

0.28 

0.95 

1.03 

1.51 

1.52 

0.28 

0.90 

0.98 

1.42 

1.43 

0.31 

0.97 

0.99 

1.43 

1.44 

0.35 

0.97 

0.99 

1.47 

1.47 

20.0 

0.31 

0.83 

0.95 

1.36 

1.36 

0.29 

0.80 

0.93 

1.37 

1.37 

0.32 

0.78 

0.92 

1.28 

1.28 

0.39 

0.92 

0.94 

1.39 

1.40 

25.0 

0.31 

0.77 

0.87 

1.26 

1.26 

0.31 

0.71 

0.84 

1.18 

1.18 

0.34 

0.73 

0.84 

1.23 

1.23 

0.40 

0.79 

0.85 

1.25 

1.25 

30.0 

0.35 

0.69 

0.78 

1.16 

1.16 

0.36 

0.75 

0.83 

1.20 

1.20 

0.38 

0.78 

0.89 

1.30 

1.30 

0.42 

0.79 

0.87 

1.26 

1.26 

35.0 

0.32 

0.60 

0.78 

1.12 

1.13 

0.32 

0.57 

0.73 

1.04 

1.05 

0.39 

0.66 

0.79 

1.13 

1.14 

0.42 

0.74 

0.84 

1.18 

1.18 

40.0 

0.33 

0.57 

0.72 

1.06 

1.06 

0.34 

0.59 

0.76 

1.07 

1.07 

0.41 

0.67 

0.83 

1.13 

1.13 

0.40 

0.67 

0.83 

1.11 

1.11 

50.0 

0.38 

0.58 

0.74 

1.04 

1.04 

0.39 

0.55 

0.70 

0.99 

0.99 

0.38 

0.57 

0.75 

1.01 

1.01 

0.43 

0.62 

0.79 

1.05 

1.05 



Table 9: Relative efficiencies of IPT, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0, p = 20, u = 10 for 
different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.06 

1.33 

1.81 

5.18 

5.21 

0.07 

1.41 

1.73 

2.63 

2.66 

0.14 

1.05 

1.22 

1.90 

1.91 

0.17 

1.07 

1.12 

1.58 

1.59 

0.1 

0.05 

1.40 

1.96 

4.58 

4.64 

0.08 

1.37 

1.70 

2.65 

2.68 

0.13 

0.95 

1.18 

1.80 

1.80 

0.16 

0.96 

1.10 

1.49 

1.49 

0.2 

0.06 

2.25 

2.94 

5.17 

5.23 

0.08 

1.12 

1.37 

2.41 

2.43 

0.13 

1.04 

1.21 

1.71 

1.72 

0.18 

0.96 

1.07 

1.66 

1.66 

0.3 

0.07 

1.16 

1.57 

3.17 

3.22 

0.08 

1.23 

1.37 

2.35 

2.36 

0.15 

1.09 

1.14 

1.84 

1.85 

0.19 

1.04 

1.14 

1.61 

1.61 

0.4 

0.07 

1.87 

2.35 

3.93 

3.96 

0.10 

1.22 

1.42 

2.47 

2.49 

0.13 

1.09 

1.24 

1.79 

1.80 

0.18 

0.95 

1.07 

1.59 

1.60 

0.5 

0.07 

1.46 

1.56 

3.25 

3.29 

0.10 

1.10 

1.36 

2.39 

2.40 

0.15 

1.08 

1.18 

1.75 

1.76 

0.19 

1.00 

1.08 

1.60 

1.61 

0.6 

0.07 

1.35 

1.48 

3.06 

3.10 

0.09 

1.43 

1.62 

2.35 

2.36 

0.15 

1.21 

1.29 

1.86 

1.87 

0.19 

0.99 

1.10 

1.61 

1.61 

0.7 

0.07 

1.12 

1.46 

2.83 

2.84 

0.09 

1.32 

1.55 

2.09 

2.11 

0.15 

1.13 

1.25 

1.72 

1.73 

0.18 

0.98 

1.10 

1.55 

1.56 

0.8 

0.05 

1.44 

2.03 

2.65 

2.67 

0.10 

1.14 

1.32 

2.02 

2.03 

0.16 

1.03 

1.16 

1.71 

1.71 

0.19 

1.01 

1.11 

1.52 

1.53 

0.9 

0.08 

1.16 

1.47 

2.73 

2.76 

0.08 

1.13 

1.35 

1.88 

1.89 

0.16 

1.16 

1.24 

1.74 

1.74 

0.19 

1.07 

1.10 

1.53 

1.53 

1.0 

0.08 

1.32 

1.89 

2.76 

2.83 

0.11 

1.28 

1.46 

2.01 

2.03 

0.15 

1.01 

1.14 

1.75 

1.75 

0.21 

0.99 

1.10 

1.57 

1.57 

1.5 

0.08 

1.03 

1.29 

2.04 

2.06 

0.11 

1.14 

1.24 

1.93 

1.94 

0.15 

0.98 

1.14 

1.57 

1.58 

0.20 

0.96 

1.08 

1.48 

1.48 

2.0 

0.13 

1.20 

1.23 

2.17 

2.18 

0.13 

1.06 

1.17 

1.71 

1.74 

0.17 

1.03 

1.11 

1.56 

1.57 

0.22 

0.95 

1.06 

1.49 

1.50 

3.0 

0.12 

1.02 

1.17 

1.76 

1.77 

0.14 

1.02 

1.13 

1.61 

1.62 

0.19 

0.98 

1.13 

1.55 

1.56 

0.22 

0.94 

1.03 

1.40 

1.40 

5.0 

0.17 

0.86 

0.99 

1.63 

1.65 

0.16 

0.80 

0.95 

1.36 

1.38 

0.20 

0.85 

0.97 

1.36 

1.36 

0.27 

0.93 

0.98 

1.40 

1.40 

10.0 

0.18 

0.58 

0.76 

1.07 

1.08 

0.20 

0.61 

0.79 

1.09 

1.09 

0.24 

0.66 

0.84 

1.14 

1.15 

0.29 

0.75 

0.89 

1.14 

1.14 

15.0 

0.18 

0.45 

0.70 

0.91 

0.91 

0.20 

0.50 

0.76 

0.98 

0.98 

0.27 

0.63 

0.83 

1.06 

1.06 

0.30 

0.62 

0.81 

1.04 

1.04 

20.0 

0.18 

0.39 

0.66 

0.78 

0.78 

0.21 

0.42 

0.71 

0.86 

0.86 

0.27 

0.52 

0.79 

0.92 

0.92 

0.29 

0.57 

0.86 

0.97 

0.97 

25.0 

0.18 

0.36 

0.76 

0.73 

0.73 

0.21 

0.41 

0.79 

0.82 

0.83 

0.27 

0.50 

0.84 

0.91 

0.91 

0.29 

0.53 

0.91 

0.93 

0.93 

30.0 

0.18 

0.33 

0.78 

0.70 

0.70 

0.21 

0.37 

0.84 

0.74 

0.74 

0.26 

0.45 

0.86 

0.84 

0.84 

0.29 

0.51 

0.95 

0.84 

0.84 

35.0 

0.20 

0.33 

0.80 

0.69 

0.69 

0.20 

0.34 

0.92 

0.72 

0.72 

0.23 

0.38 

0.93 

0.74 

0.74 

0.31 

0.52 

0.96 

0.90 

0.90 

40.0 

0.17 

0.29 

0.92 

0.68 

0.68 

0.22 

0.36 

0.94 

0.75 

0.75 

0.24 

0.38 

0.93 

0.76 

0.76 

0.29 

0.46 

0.98 

0.84 

0.84 

50.0 

0.17 

0.27 

0.93 

0.65 

0.65 

0.19 

0.29 

1.00 

0.67 

0.67 

0.25 

0.38 

0.96 

0.77 

0.77 

0.32 

0.47 

0.96 

0.84 

0.84 



Table 10; Relative efficiencies of IPT, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0, p = 20, a = 20 for 
different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.07 

1.75 

2.13 

5.17 

5.26 

0.06 

1.09 

1.86 

4.01 

4.23 

0.06 

1.40 

1.56 

2.69 

2.73 

0.08 

1.05 

1.42 

2.36 

2.38 

0.1 

0.04 

1.48 

2.53 

5.89 

6.28 

0.06 

1.59 

2.09 

3.55 

3.56 

0.06 

1.65 

1.92 

2.71 

2.72 

0.09 

1.26 

1.30 

2.39 

2.42 

0.2 

0.06 

1.57 

2.04 

4.99 

5.03 

0.05 

1.39 

2.26 

4.20 

4.27 

0.07 

1.72 

1.83 

2.80 

2.84 

0.08 

1.06 

1.35 

2.35 

2.37 

0.3 

0.04 

1.39 

2.53 

5.85 

5.89 

0.07 

1.42 

1.64 

3.92 

3.98 

0.06 

1.24 

1.71 

2.48 

2.49 

0.09 

1.37 

1.60 

2.48 

2.49 

0.4 

0.05 

1.51 

2.63 

5.83 

6.06 

0.07 

1.52 

1.87 

3.60 

3.61 

0.07 

1.17 

1.67 

2.88 

2.91 

0.08 

1.12 

1.57 

2.45 

2.46 

0.5 

0.06 

1.75 

2.42 

5.23 

5.28 

0.06 

1.66 

1.83 

3.65 

3.72 

0.07 

1.46 

1.77 

2.84 

2.86 

0.07 

1.06 

1.35 

1.98 

1.99 

0.6 

0.05 

1.47 

2.21 

4.55 

4.68 

0.06 

1.08 

1.54 

3.56 

3.60 

0.08 

1.59 

1.61 

2.69 

2.72 

0.09 

1.30 

1.45 

2.26 

2.28 

0.7 

0.06 

1.56 

1.64 

4.16 

4.21 

0.05 

1.54 

2.21 

3.72 

3.77 

0.06 

1.14 

1.55 

2.27 

2.31 

0.09 

1.31 

1.42 

2.35 

2.38 

0.8 

0.06 

1.65 

2.16 

4.90 

4.96 

0.05 

1.37 

1.87 

3.47 

3.49 

0.07 

1.33 

1.83 

2.70 

2.74 

0.09 

1.29 

1.59 

2.38 

2.40 

0.9 

0.04 

1.84 

3.65 

4.83 

5.08 

0.07 

1.15 

1.90 

3.28 

3.32 

0.08 

1.52 

1.92 

2.97 

3.01 

0.09 

1.46 

1.55 

2.35 

2.38 

1.0 

0.06 

1.42 

1.73 

3.71 

3.76 

0.06 

1.23 

1.86 

3.16 

3.20 

0.07 

1.28 

1.61 

2.41 

2.42 

0.09 

1.12 

1.40 

2.25 

2.26 

1.5 

0.07 

1.43 

1.66 

3.51 

3.57 

0.08 

1.39 

1.69 

3.22 

3.26 

0.06 

1.06 

1.62 

2.27 

2.29 

0.09 

1.06 

1.34 

2.06 

2.08 

2.0 

0.07 

1.51 

1.54 

3.05 

3.07 

0.07 

1.31 

1.68 

2.93 

2.97 

0.09 

1.27 

1.40 

2.35 

2.37 

0.09 

1.09 

1.27 

1.94 

1.96 

3.0 

0.06 

1.21 

1.69 

2.61 

2.64 

0.08 

1.14 

1.45 

2.60 

2.61 

0.10 

1.30 

1.32 

2.20 

2.21 

0.10 

1.10 

1.21 

2.07 

2.07 

5.0 

0.09 

1.48 

1.59 

2.51 

2.53 

0.10 

1.10 

1.32 

2.23 

2.25 

0.10 

1.09 

1.29 

1.95 

1.96 

0.11 

1.13 

1.37 

1.96 

1.98 

10.0 

0.09 

1.00 

1.27 

1.75 

1.78 

0.12 

1.08 

1.32 

1.94 

1.96 

0.13 

1.02 

1.12 

1.71 

1.72 

0.15 

1.05 

1.17 

1.73 

1.74 

15.0 

0.13 

0.88 

1.10 

1.58 

1.61 

0.13 

0.90 

1.08 

1.52 

1.52 

0.14 

0.95 

1.08 

1.51 

1.53 

0.17 

1.00 

1.05 

1.56 

1.57 

20.0 

0.14 

0.87 

0.98 

1.45 

1.46 

0.16 

0.91 

1.04 

1.49 

1.50 

0.18 

0.88 

1.01 

1.57 

1.58 

0.18 

0.89 

1.00 

1.48 

1.49 

25.0 

0.14 

0.76 

0.92 

1.33 

1.33 

0.15 

0.72 

0.95 

1.35 

1.36 

0.16 

0.75 

0.94 

1.26 

1.27 

0.21 

0.88 

0.98 

1.42 

1.44 

30.0 

0.18 

0.75 

0.89 

1.34 

1.35 

0.17 

0.69 

0.88 

1.29 

1.30 

0.19 

0.70 

0.89 

1.30 

1.30 

0.19 

0.73 

0.92 

1.24 

1.24 

35.0 

0.18 

0.67 

0.82 

1.15 

1.16 

0.17 

0.63 

0.81 

1.10 

1.10 

0.19 

0.69 

0.84 

1.15 

1.16 

0.19 

0.66 

0.85 

1.16 

1.17 

40.0 

0.19 

0.59 

0.78 

1.23 

1.23 

0.18 

0.62 

0.80 

1.09 

1.10 

0.20 

0.62 

0.82 

1.12 

1.13 

0.21 

0.64 

0.83 

1.16 

1.17 

50.0 

0.17 

0.47 

0.67 

0.88 

0.88 

0.18 

0.51 

0.72 

0.99 

0.99 

0.20 

0.54 

0.75 

1.04 

1.04 

0.21 

0.57 

0.79 

1.01 

1.01 


Table 11; Relative efficiencies of IPT, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0, p = 50, a = 10 for 
different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.02 

0.57 

2.24 

9.33 

10.34 

0.02 

0.71 

1.52 

3.61 

3.68 

0.04 

0.70 

1.39 

2.18 

2.20 

0.04 

0.66 

1.20 

1.68 

1.69 

0.1 

0.02 

0.62 

1.94 

7.52 

8.26 

0.02 

0.58 

1.44 

3.04 

3.07 

0.04 

0.68 

1.27 

2.13 

2.15 

0.05 

0.71 

1.23 

1.86 

1.87 

0.2 

0.02 

0.58 

1.63 

5.86 

6.17 

0.02 

0.65 

1.47 

2.87 

2.99 

0.03 

0.71 

1.32 

1.94 

1.97 

0.05 

0.75 

1.18 

1.73 

1.75 

0.3 

0.02 

0.57 

1.75 

4.63 

4.76 

0.03 

0.57 

1.44 

2.93 

2.97 

0.03 

0.66 

1.29 

1.86 

1.88 

0.05 

0.64 

1.16 

1.71 

1.72 

0.4 

0.02 

0.94 

2.03 

5.59 

5.84 

0.02 

0.49 

1.46 

2.57 

2.61 

0.04 

0.59 

1.34 

1.98 

2.00 

0.05 

0.76 

1.18 

1.82 

1.82 

0.5 

0.02 

0.78 

2.19 

4.93 

5.05 

0.03 

0.92 

1.49 

3.23 

3.28 

0.03 

0.66 

1.24 

1.82 

1.84 

0.05 

0.76 

1.20 

1.77 

1.79 

0.6 

0.02 

0.69 

1.98 

4.24 

4.36 

0.02 

0.63 

1.75 

2.44 

2.49 

0.04 

0.79 

1.38 

2.05 

2.07 

0.05 

0.64 

1.10 

1.72 

1.73 

0.7 

0.02 

0.43 

1.47 

3.58 

3.67 

0.02 

0.68 

1.36 

2.28 

2.30 

0.03 

0.75 

1.36 

1.84 

1.87 

0.05 

0.63 

1.20 

1.68 

1.68 

0.8 

0.02 

0.51 

1.59 

3.63 

3.77 

0.02 

0.71 

1.70 

2.32 

2.35 

0.04 

0.63 

1.36 

1.83 

1.85 

0.05 

0.71 

1.22 

1.67 

1.68 

0.9 

0.02 

0.79 

2.14 

3.61 

3.69 

0.03 

0.63 

1.37 

2.31 

2.35 

0.04 

0.64 

1.27 

1.86 

1.87 

0.05 

0.64 

1.15 

1.62 

1.63 

1.0 

0.02 

0.69 

1.71 

3.31 

3.36 

0.03 

0.64 

1.28 

2.69 

2.74 

0.04 

0.67 

1.22 

1.71 

1.72 

0.05 

0.63 

1.14 

1.56 

1.57 

1.5 

0.02 

0.54 

1.56 

2.49 

2.51 

0.03 

0.74 

1.45 

2.29 

2.31 

0.04 

0.73 

1.28 

1.69 

1.70 

0.05 

0.74 

1.13 

1.59 

1.60 

2.0 

0.03 

0.76 

1.50 

2.53 

2.58 

0.03 

0.67 

1.21 

2.07 

2.08 

0.05 

0.72 

1.25 

1.73 

1.74 

0.06 

0.65 

1.14 

1.59 

1.60 

3.0 

0.03 

0.70 

1.31 

2.00 

2.04 

0.04 

0.64 

1.27 

1.96 

1.98 

0.05 

0.66 

1.14 

1.69 

1.70 

0.06 

0.70 

1.16 

1.47 

1.48 

5.0 

0.04 

0.62 

1.11 

1.57 

1.59 

0.05 

0.65 

1.13 

1.59 

1.60 

0.05 

0.64 

1.09 

1.41 

1.43 

0.07 

0.63 

1.06 

1.40 

1.41 

10.0 

0.06 

0.56 

0.88 

1.22 

1.25 

0.05 

0.55 

0.93 

1.12 

1.15 

0.07 

0.53 

0.90 

1.20 

1.22 

0.08 

0.52 

0.91 

1.19 

1.20 

15.0 

0.06 

0.40 

0.73 

0.95 

0.97 

0.07 

0.44 

0.79 

1.08 

1.09 

0.08 

0.48 

0.80 

1.06 

1.06 

0.08 

0.50 

0.82 

1.01 

1.02 

20.0 

0.06 

0.30 

0.65 

0.80 

0.81 

0.06 

0.32 

0.67 

0.74 

0.76 

0.07 

0.35 

0.68 

0.82 

0.83 

0.08 

0.39 

0.77 

0.92 

0.92 

25.0 

0.06 

0.27 

0.64 

0.72 

0.73 

0.07 

0.30 

0.65 

0.74 

0.75 

0.08 

0.34 

0.73 

0.84 

0.85 

0.08 

0.33 

0.76 

0.83 

0.83 

30.0 

0.05 

0.21 

0.62 

0.61 

0.62 

0.06 

0.22 

0.66 

0.65 

0.66 

0.07 

0.28 

0.72 

0.71 

0.72 

0.09 

0.31 

0.73 

0.78 

0.79 

35.0 

0.06 

0.22 

0.58 

0.58 

0.59 

0.07 

0.24 

0.67 

0.67 

0.67 

0.07 

0.25 

0.65 

0.64 

0.64 

0.09 

0.29 

0.79 

0.78 

0.78 

40.0 

0.06 

0.17 

0.60 

0.54 

0.55 

0.06 

0.18 

0.63 

0.56 

0.56 

0.08 

0.24 

0.72 

0.66 

0.66 

0.08 

0.25 

0.78 

0.70 

0.70 

50.0 

0.07 

0.18 

0.68 

0.55 

0.55 

0.05 

0.15 

0.69 

0.49 

0.49 

0.08 

0.22 

0.80 

0.63 

0.63 

0.08 

0.22 

0.83 

0.63 

0.63 


Table 12; Relative efficiencies of IPT, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0, p = 50, a = 20 for 
different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.02 

0.50 

1.56 

6.11 

6.32 

0.01 

0.61 

1.79 

5.22 

5.42 

0.02 

0.55 

1.50 

3.68 

3.74 

0.02 

0.67 

1.42 

2.67 

2.75 

0.1 

0.02 

0.74 

1.53 

6.18 

6.45 

0.02 

0.50 

2.50 

6.17 

6.36 

0.02 

0.71 

1.95 

4.01 

4.12 

0.02 

0.53 

1.59 

2.75 

2.82 

0.2 

0.02 

0.74 

2.09 

9.49 

9.72 

0.02 

0.65 

2.73 

6.12 

6.26 

0.02 

0.47 

1.61 

3.25 

3.29 

0.03 

0.88 

1.84 

3.04 

3.11 

0.3 

0.02 

0.67 

1.76 

7.36 

7.74 

0.01 

0.40 

1.90 

5.15 

5.38 

0.02 

0.68 

1.69 

3.72 

3.84 

0.02 

0.73 

1.73 

2.94 

3.02 

0.4 

0.02 

0.46 

1.45 

5.86 

5.97 

0.02 

0.58 

1.88 

4.64 

4.76 

0.01 

0.49 

1.73 

3.10 

3.18 

0.03 

0.87 

1.49 

3.38 

3.43 

0.5 

0.01 

0.61 

2.19 

6.87 

7.68 

0.02 

0.56 

1.54 

4.56 

4.71 

0.02 

0.71 

1.83 

3.33 

3.44 

0.02 

0.74 

1.56 

2.84 

2.90 

0.6 

0.02 

0.59 

1.68 

5.96 

6.21 

0.02 

0.75 

1.89 

4.26 

4.48 

0.03 

0.62 

1.76 

3.83 

3.94 

0.02 

0.50 

1.51 

2.82 

2.84 

0.7 

0.02 

0.75 

1.82 

5.28 

5.61 

0.02 

0.74 

1.99 

5.19 

5.50 

0.02 

0.49 

1.47 

2.89 

3.01 

0.03 

0.61 

1.48 

2.98 

3.01 

0.8 

0.02 

0.59 

2.27 

6.12 

6.60 

0.02 

0.69 

2.37 

5.38 

5.53 

0.02 

0.69 

1.65 

3.56 

3.71 

0.02 

0.60 

1.51 

2.67 

2.68 

0.9 

0.02 

0.80 

4.01 

7.71 

8.17 

0.02 

0.85 

1.94 

4.64 

4.86 

0.02 

0.49 

1.50 

3.22 

3.27 

0.02 

0.71 

1.66 

2.80 

2.84 

1.0 

0.02 

0.49 

1.56 

5.71 

5.87 

0.02 

0.58 

1.64 

4.45 

4.60 

0.02 

0.55 

1.54 

3.17 

3.26 

0.03 

0.68 

1.44 

2.79 

2.81 

1.5 

0.02 

0.65 

2.13 

4.96 

5.08 

0.02 

0.56 

1.61 

3.96 

4.00 

0.03 

0.80 

1.56 

3.27 

3.34 

0.03 

0.58 

1.30 

2.67 

2.69 

2.0 

0.02 

0.72 

1.43 

4.07 

4.14 

0.02 

0.51 

1.61 

3.33 

3.42 

0.02 

0.63 

1.45 

3.02 

3.05 

0.03 

0.65 

1.52 

2.69 

2.72 

3.0 

0.02 

0.62 

1.49 

3.96 

4.06 

0.02 

0.67 

1.58 

3.19 

3.25 

0.02 

0.63 

1.66 

2.70 

2.78 

0.04 

0.71 

1.45 

2.87 

2.91 

5.0 

0.02 

0.68 

1.44 

2.55 

2.61 

0.02 

0.63 

1.70 

2.37 

2.41 

0.03 

0.68 

1.43 

2.60 

2.63 

0.03 

0.72 

1.39 

2.33 

2.37 

10.0 

0.03 

0.76 

1.29 

2.36 

2.39 

0.03 

0.60 

1.33 

1.99 

2.06 

0.04 

0.70 

1.45 

2.03 

2.08 

0.04 

0.62 

1.29 

1.74 

1.77 

15.0 

0.03 

0.55 

1.20 

1.71 

1.73 

0.04 

0.64 

1.15 

1.82 

1.84 

0.04 

0.53 

1.16 

1.69 

1.71 

0.04 

0.66 

1.15 

1.60 

1.63 

20.0 

0.04 

0.61 

1.19 

1.64 

1.67 

0.04 

0.65 

1.13 

1.61 

1.62 

0.05 

0.66 

1.13 

1.68 

1.69 

0.04 

0.59 

1.07 

1.41 

1.43 

25.0 

0.04 

0.60 

1.11 

1.45 

1.49 

0.05 

0.66 

1.07 

1.49 

1.53 

0.05 

0.59 

1.01 

1.45 

1.47 

0.05 

0.54 

1.04 

1.52 

1.53 

30.0 

0.05 

0.49 

0.95 

1.29 

1.33 

0.05 

0.64 

1.03 

1.43 

1.44 

0.05 

0.54 

0.99 

1.38 

1.40 

0.06 

0.54 

0.98 

1.38 

1.40 

35.0 

0.05 

0.53 

0.94 

1.22 

1.26 

0.05 

0.47 

0.91 

1.16 

1.18 

0.05 

0.47 

0.92 

1.19 

1.21 

0.05 

0.59 

0.95 

1.16 

1.19 

40.0 

0.05 

0.43 

0.82 

1.05 

1.09 

0.05 

0.42 

0.87 

1.12 

1.14 

0.05 

0.50 

0.90 

1.12 

1.13 

0.06 

0.55 

0.91 

1.16 

1.18 

50.0 

0.06 

0.50 

0.84 

1.18 

1.19 

0.06 

0.42 

0.83 

1.05 

1.07 

0.07 

0.47 

0.86 

1.20 

1.22 

0.06 

0.44 

0.79 

0.96 

0.98 



Table 13: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.5, p = 10, a = 10 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.08 

1.98 

2.23 

3.66 

3.67 

0.14 

1.27 

1.54 

2.30 

2.31 

0.22 

0.83 

1.12 

1.79 

1.79 

0.32 

0.67 

1.01 

1.60 

1.60 

0.1 

0.09 

1.49 

1.86 

2.94 

2.99 

0.14 

1.20 

1.43 

2.26 

2.27 

0.24 

0.83 

1.11 

1.81 

1.81 

0.33 

0.64 

1.01 

1.55 

1.55 

0.2 

0.10 

1.75 

1.90 

2.86 

2.87 

0.13 

1.16 

1.47 

2.25 

2.26 

0.24 

0.79 

1.10 

1.76 

1.77 

0.31 

0.63 

1.00 

1.56 

1.56 

0.3 

0.10 

1.73 

2.24 

3.02 

3.03 

0.15 

1.21 

1.46 

2.20 

2.21 

0.25 

0.84 

1.10 

1.78 

1.78 

0.34 

0.64 

0.99 

1.54 

1.54 

0.4 

0.11 

1.52 

1.75 

2.73 

2.74 

0.16 

1.17 

1.45 

2.25 

2.26 

0.26 

0.78 

1.08 

1.76 

1.76 

0.33 

0.63 

1.00 

1.54 

1.54 

0.5 

0.11 

1.36 

1.67 

2.65 

2.66 

0.15 

1.07 

1.35 

2.06 

2.07 

0.25 

0.77 

1.10 

1.76 

1.76 

0.34 

0.65 

1.01 

1.56 

1.56 

0.6 

0.10 

1.35 

1.69 

2.53 

2.54 

0.16 

1.11 

1.44 

2.07 

2.08 

0.24 

0.66 

1.06 

1.70 

1.71 

0.35 

0.65 

1.00 

1.54 

1.54 

0.7 

0.14 

1.38 

1.58 

2.50 

2.51 

0.15 

1.01 

1.35 

2.00 

2.01 

0.27 

0.75 

1.07 

1.71 

1.71 

0.35 

0.63 

1.01 

1.52 

1.52 

0.8 

0.13 

1.36 

1.60 

2.43 

2.44 

0.16 

1.04 

1.35 

1.97 

1.98 

0.28 

0.76 

1.06 

1.69 

1.69 

0.35 

0.64 

1.00 

1.53 

1.53 

0.9 

0.12 

1.28 

1.46 

2.25 

2.26 

0.18 

1.08 

1.31 

2.06 

2.07 

0.26 

0.71 

1.06 

1.69 

1.69 

0.36 

0.63 

1.00 

1.50 

1.50 

1.0 

0.15 

1.21 

1.34 

2.16 

2.17 

0.18 

1.03 

1.28 

1.92 

1.93 

0.27 

0.72 

1.04 

1.67 

1.68 

0.35 

0.62 

0.99 

1.48 

1.48 

1.5 

0.16 

1.20 

1.36 

2.14 

2.14 

0.20 

0.99 

1.20 

1.88 

1.88 

0.27 

0.69 

1.04 

1.62 

1.62 

0.39 

0.67 

0.99 

1.47 

1.48 

2.0 

0.16 

1.03 

1.22 

1.82 

1.84 

0.19 

0.90 

1.22 

1.82 

1.83 

0.28 

0.67 

1.02 

1.60 

1.60 

0.39 

0.66 

1.00 

1.45 

1.45 

3.0 

0.18 

0.95 

1.21 

1.75 

1.76 

0.23 

0.83 

1.08 

1.66 

1.66 

0.31 

0.68 

0.99 

1.54 

1.54 

0.41 

0.66 

1.00 

1.41 

1.41 

5.0 

0.21 

0.79 

1.05 

1.51 

1.52 

0.25 

0.69 

0.98 

1.47 

1.47 

0.36 

0.67 

0.97 

1.44 

1.44 

0.41 

0.62 

0.99 

1.35 

1.35 

10.0 

0.26 

0.58 

0.86 

1.23 

1.23 

0.30 

0.57 

0.87 

1.25 

1.25 

0.38 

0.60 

0.98 

1.30 

1.30 

0.43 

0.59 

1.00 

1.24 

1.24 

15.0 

0.29 

0.51 

0.82 

1.11 

1.11 

0.32 

0.52 

0.92 

1.18 

1.18 

0.39 

0.57 

0.98 

1.21 

1.21 

0.45 

0.59 

1.00 

1.19 

1.19 

20.0 

0.29 

0.46 

0.85 

1.03 

1.03 

0.32 

0.49 

0.91 

1.09 

1.09 

0.37 

0.50 

0.99 

1.14 

1.14 

0.45 

0.57 

1.00 

1.16 

1.16 

25.0 

0.29 

0.43 

0.89 

0.99 

0.99 

0.30 

0.43 

0.93 

1.02 

1.02 

0.38 

0.50 

0.99 

1.11 

1.11 

0.46 

0.57 

1.00 

1.14 

1.14 

30.0 

0.30 

0.43 

0.94 

0.98 

0.98 

0.34 

0.46 

0.97 

1.03 

1.03 

0.40 

0.51 

1.00 

1.09 

1.09 

0.45 

0.54 

1.00 

1.11 

1.11 

35.0 

0.30 

0.41 

0.98 

0.96 

0.96 

0.32 

0.42 

0.99 

1.00 

1.00 

0.37 

0.46 

1.00 

1.07 

1.07 

0.44 

0.53 

1.00 

1.09 

1.09 

40.0 

0.29 

0.38 

0.97 

0.93 

0.93 

0.31 

0.39 

0.99 

0.96 

0.96 

0.39 

0.48 

1.00 

1.06 

1.06 

0.44 

0.52 

1.00 

1.08 

1.08 

50.0 

0.31 

0.40 

1.00 

0.93 

0.93 

0.32 

0.39 

1.00 

0.93 

0.93 

0.38 

0.46 

1.00 

1.04 

1.04 

0.46 

0.53 

1.00 

1.07 

1.07 



Table 14: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.5, p = 10, a = 20 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.10 

1.62 

1.91 

3.14 

3.14 

0.09 

1.47 

1.69 

2.75 

2.78 

0.13 

1.15 

1.58 

2.49 

2.50 

0.16 

0.93 

1.30 

2.15 

2.16 

0.1 

0.09 

1.59 

1.88 

3.35 

3.36 

0.11 

2.13 

2.45 

3.09 

3.10 

0.13 

1.07 

1.56 

2.38 

2.38 

0.17 

0.89 

1.30 

2.14 

2.15 

0.2 

0.11 

1.67 

1.87 

3.06 

3.06 

0.12 

1.62 

1.89 

2.90 

2.90 

0.13 

1.23 

1.74 

2.48 

2.49 

0.16 

0.93 

1.37 

2.19 

2.19 

0.3 

0.11 

1.86 

1.97 

3.06 

3.09 

0.09 

1.55 

1.72 

2.80 

2.81 

0.13 

1.12 

1.49 

2.33 

2.33 

0.17 

0.96 

1.38 

2.18 

2.18 

0.4 

0.09 

1.83 

2.26 

3.22 

3.24 

0.10 

1.42 

1.85 

2.78 

2.80 

0.13 

1.26 

1.62 

2.41 

2.41 

0.19 

0.98 

1.30 

2.14 

2.15 

0.5 

0.10 

1.65 

1.76 

2.94 

2.94 

0.11 

1.51 

1.81 

2.73 

2.74 

0.14 

1.10 

1.46 

2.35 

2.36 

0.19 

0.98 

1.27 

2.20 

2.21 

0.6 

0.12 

1.71 

1.83 

2.96 

2.97 

0.10 

1.43 

1.80 

2.76 

2.77 

0.14 

1.10 

1.53 

2.37 

2.38 

0.18 

0.92 

1.35 

2.11 

2.12 

0.7 

0.11 

1.68 

1.72 

2.82 

2.83 

0.11 

1.24 

1.57 

2.52 

2.53 

0.14 

1.09 

1.48 

2.37 

2.37 

0.18 

0.87 

1.33 

2.11 

2.12 

0.8 

0.10 

2.00 

2.23 

3.29 

3.29 

0.12 

1.50 

1.70 

2.63 

2.64 

0.14 

1.22 

1.61 

2.43 

2.43 

0.17 

0.89 

1.33 

2.13 

2.14 

0.9 

0.09 

1.55 

1.85 

2.91 

2.92 

0.11 

1.84 

2.00 

2.74 

2.76 

0.15 

1.17 

1.52 

2.35 

2.35 

0.19 

0.87 

1.27 

2.14 

2.14 

1.0 

0.09 

1.69 

2.34 

3.00 

3.02 

0.12 

1.36 

1.58 

2.55 

2.56 

0.15 

1.10 

1.37 

2.10 

2.11 

0.19 

0.93 

1.30 

2.17 

2.17 

1.5 

0.09 

1.91 

2.31 

2.98 

3.00 

0.13 

1.40 

1.71 

2.61 

2.62 

0.15 

1.02 

1.45 

2.29 

2.30 

0.20 

0.91 

1.28 

2.13 

2.13 

2.0 

0.13 

1.56 

1.60 

2.58 

2.58 

0.14 

1.35 

1.46 

2.42 

2.43 

0.17 

1.03 

1.39 

2.16 

2.16 

0.21 

0.89 

1.24 

2.06 

2.06 

3.0 

0.11 

1.49 

1.81 

2.56 

2.57 

0.13 

1.23 

1.59 

2.31 

2.31 

0.17 

1.09 

1.38 

2.17 

2.18 

0.22 

0.84 

1.17 

1.97 

1.98 

5.0 

0.14 

1.28 

1.46 

2.16 

2.17 

0.16 

1.18 

1.47 

2.18 

2.19 

0.18 

0.91 

1.26 

2.00 

2.01 

0.22 

0.76 

1.15 

1.90 

1.91 

10.0 

0.16 

0.99 

1.23 

1.79 

1.80 

0.19 

0.93 

1.22 

1.83 

1.83 

0.21 

0.82 

1.14 

1.78 

1.78 

0.25 

0.68 

1.03 

1.69 

1.70 

15.0 

0.20 

0.89 

1.12 

1.67 

1.67 

0.19 

0.80 

1.08 

1.57 

1.58 

0.25 

0.71 

1.04 

1.62 

1.62 

0.26 

0.61 

1.01 

1.58 

1.59 

20.0 

0.24 

0.86 

1.06 

1.59 

1.59 

0.23 

0.74 

1.00 

1.54 

1.55 

0.26 

0.67 

1.00 

1.53 

1.53 

0.30 

0.63 

0.97 

1.50 

1.50 

25.0 

0.26 

0.79 

0.98 

1.49 

1.49 

0.27 

0.73 

0.96 

1.49 

1.49 

0.31 

0.68 

0.96 

1.51 

1.51 

0.30 

0.59 

0.96 

1.45 

1.45 

30.0 

0.26 

0.72 

0.94 

1.35 

1.35 

0.27 

0.64 

0.93 

1.38 

1.38 

0.29 

0.59 

0.92 

1.38 

1.38 

0.31 

0.55 

0.94 

1.36 

1.36 

35.0 

0.26 

0.62 

0.90 

1.30 

1.30 

0.28 

0.62 

0.90 

1.32 

1.32 

0.29 

0.57 

0.91 

1.34 

1.34 

0.33 

0.57 

0.95 

1.37 

1.37 

40.0 

0.27 

0.58 

0.86 

1.24 

1.25 

0.28 

0.57 

0.85 

1.24 

1.24 

0.31 

0.56 

0.91 

1.28 

1.28 

0.31 

0.52 

0.94 

1.28 

1.28 

50.0 

0.28 

0.53 

0.82 

1.19 

1.19 

0.28 

0.50 

0.81 

1.14 

1.14 

0.31 

0.54 

0.91 

1.24 

1.24 

0.33 

0.53 

0.96 

1.27 

1.27 



Table 15: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.5, p = 20, ct = 10 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.04 

1.50 

2.19 

5.71 

5.77 

0.07 

1.33 

1.93 

3.48 

3.51 

0.10 

0.83 

1.31 

2.25 

2.27 

0.14 

0.57 

1.10 

1.96 

1.96 

0.1 

0.05 

1.31 

1.86 

4.35 

4.39 

0.06 

1.31 

2.27 

3.27 

3.31 

0.11 

0.78 

1.26 

2.23 

2.25 

0.15 

0.55 

1.09 

1.99 

2.00 

0.2 

0.05 

1.29 

2.04 

4.64 

4.69 

0.07 

1.01 

1.59 

2.92 

2.95 

0.12 

0.76 

1.22 

2.36 

2.37 

0.16 

0.58 

1.07 

1.95 

1.95 

0.3 

0.06 

1.57 

2.26 

4.55 

4.59 

0.07 

1.02 

1.51 

2.90 

2.92 

0.11 

0.78 

1.28 

2.25 

2.26 

0.16 

0.56 

1.04 

1.96 

1.96 

0.4 

0.06 

1.38 

1.86 

4.10 

4.17 

0.07 

0.99 

1.60 

2.85 

2.88 

0.12 

0.83 

1.26 

2.31 

2.32 

0.16 

0.53 

1.07 

1.92 

1.92 

0.5 

0.05 

0.95 

1.62 

3.56 

3.64 

0.07 

1.01 

1.51 

2.68 

2.70 

0.12 

0.75 

1.20 

2.18 

2.19 

0.16 

0.53 

1.05 

1.91 

1.91 

0.6 

0.05 

1.10 

1.67 

3.45 

3.47 

0.08 

1.11 

1.52 

2.79 

2.83 

0.12 

0.73 

1.19 

2.24 

2.25 

0.17 

0.52 

1.04 

1.94 

1.94 

0.7 

0.05 

1.06 

1.62 

3.16 

3.18 

0.08 

0.98 

1.47 

2.64 

2.66 

0.12 

0.73 

1.20 

2.19 

2.19 

0.18 

0.60 

1.05 

1.94 

1.95 

0.8 

0.05 

1.03 

1.70 

3.32 

3.35 

0.09 

1.08 

1.39 

2.59 

2.61 

0.13 

0.73 

1.18 

2.17 

2.18 

0.17 

0.53 

1.04 

1.90 

1.90 

0.9 

0.06 

1.21 

1.78 

3.21 

3.25 

0.08 

0.97 

1.50 

2.58 

2.59 

0.12 

0.68 

1.26 

2.20 

2.21 

0.18 

0.56 

1.04 

1.89 

1.89 

1.0 

0.07 

1.25 

1.61 

3.16 

3.17 

0.09 

1.10 

1.45 

2.61 

2.62 

0.13 

0.73 

1.19 

2.16 

2.16 

0.17 

0.52 

1.05 

1.90 

1.90 

1.5 

0.06 

1.19 

1.56 

2.58 

2.60 

0.09 

1.02 

1.30 

2.32 

2.33 

0.13 

0.66 

1.14 

2.04 

2.04 

0.18 

0.51 

1.03 

1.83 

1.83 

2.0 

0.07 

1.00 

1.58 

2.48 

2.51 

0.10 

0.93 

1.28 

2.18 

2.20 

0.14 

0.63 

1.11 

2.00 

2.00 

0.18 

0.51 

1.02 

1.79 

1.79 

3.0 

0.09 

1.04 

1.33 

2.11 

2.13 

0.11 

0.89 

1.29 

2.06 

2.08 

0.15 

0.60 

1.07 

1.89 

1.89 

0.20 

0.51 

1.01 

1.76 

1.76 

5.0 

0.11 

0.84 

1.17 

1.77 

1.78 

0.12 

0.73 

1.10 

1.75 

1.76 

0.19 

0.59 

1.02 

1.75 

1.75 

0.20 

0.47 

1.00 

1.59 

1.59 

10.0 

0.14 

0.63 

0.91 

1.42 

1.43 

0.15 

0.52 

0.92 

1.42 

1.42 

0.19 

0.47 

0.96 

1.46 

1.46 

0.22 

0.44 

0.99 

1.43 

1.43 

15.0 

0.14 

0.45 

0.83 

1.15 

1.15 

0.16 

0.44 

0.87 

1.23 

1.23 

0.19 

0.41 

0.95 

1.31 

1.31 

0.24 

0.42 

1.00 

1.34 

1.34 

20.0 

0.15 

0.40 

0.81 

1.05 

1.05 

0.17 

0.39 

0.91 

1.16 

1.16 

0.18 

0.36 

0.98 

1.21 

1.21 

0.24 

0.40 

1.00 

1.26 

1.26 

25.0 

0.15 

0.34 

0.84 

1.01 

1.01 

0.17 

0.35 

0.93 

1.06 

1.06 

0.20 

0.36 

0.99 

1.18 

1.18 

0.23 

0.37 

1.00 

1.21 

1.21 

30.0 

0.14 

0.30 

0.85 

0.88 

0.88 

0.17 

0.33 

0.94 

1.03 

1.03 

0.20 

0.34 

0.99 

1.12 

1.12 

0.23 

0.34 

1.00 

1.18 

1.18 

35.0 

0.15 

0.29 

0.91 

0.89 

0.89 

0.15 

0.28 

0.96 

0.92 

0.92 

0.19 

0.31 

1.00 

1.06 

1.06 

0.23 

0.34 

1.00 

1.14 

1.14 

40.0 

0.14 

0.26 

0.92 

0.83 

0.83 

0.17 

0.30 

0.99 

0.94 

0.94 

0.20 

0.31 

1.00 

1.07 

1.07 

0.24 

0.33 

1.00 

1.11 

1.11 

50.0 

0.16 

0.26 

0.96 

0.85 

0.85 

0.17 

0.26 

0.99 

0.88 

0.88 

0.19 

0.28 

1.00 

1.04 

1.04 

0.25 

0.34 

1.00 

1.09 

1.09 



Table 16: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.5, p = 20, a = 20 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.05 

1.46 

1.92 

5.45 

5.51 

0.06 

1.32 

1.69 

4.21 

4.25 

0.06 

1.16 

1.72 

3.54 

3.57 

0.08 

0.95 

1.57 

3.04 

3.05 

0.1 

0.04 

1.67 

3.53 

7.06 

7.25 

0.05 

1.54 

2.04 

4.36 

4.43 

0.08 

1.23 

1.49 

3.28 

3.30 

0.08 

0.95 

1.60 

3.06 

3.08 

0.2 

0.05 

1.39 

1.79 

4.43 

4.50 

0.05 

1.08 

1.78 

4.14 

4.20 

0.06 

1.09 

1.90 

3.67 

3.71 

0.08 

0.88 

1.55 

3.10 

3.13 

0.3 

0.05 

1.54 

2.60 

5.84 

5.91 

0.05 

1.33 

1.93 

4.31 

4.35 

0.07 

1.16 

1.59 

3.33 

3.36 

0.08 

0.90 

1.47 

2.88 

2.90 

0.4 

0.05 

1.66 

2.63 

5.29 

5.43 

0.06 

1.28 

1.74 

4.21 

4.26 

0.07 

1.23 

1.67 

3.32 

3.38 

0.09 

1.01 

1.44 

3.02 

3.04 

0.5 

0.05 

1.25 

2.11 

4.74 

4.88 

0.06 

1.62 

1.87 

4.37 

4.40 

0.07 

1.18 

1.75 

3.48 

3.50 

0.08 

0.89 

1.53 

2.94 

2.97 

0.6 

0.05 

1.89 

2.52 

5.49 

5.57 

0.05 

1.12 

1.91 

4.08 

4.15 

0.08 

1.07 

1.60 

3.32 

3.34 

0.08 

0.94 

1.52 

3.01 

3.04 

0.7 

0.05 

1.24 

2.09 

4.63 

4.68 

0.07 

1.39 

1.78 

3.92 

3.95 

0.06 

1.04 

1.63 

3.26 

3.30 

0.09 

0.91 

1.46 

2.92 

2.94 

0.8 

0.05 

1.39 

1.87 

4.36 

4.45 

0.05 

1.21 

1.97 

4.03 

4.08 

0.06 

1.09 

1.86 

3.28 

3.32 

0.09 

0.96 

1.44 

2.88 

2.91 

0.9 

0.04 

1.35 

2.28 

4.67 

4.73 

0.07 

1.35 

1.62 

3.64 

3.68 

0.06 

1.00 

1.72 

3.22 

3.25 

0.09 

0.88 

1.53 

3.00 

3.02 

1.0 

0.05 

1.31 

1.85 

4.07 

4.11 

0.05 

1.41 

1.91 

3.90 

3.97 

0.08 

1.24 

1.55 

2.99 

3.03 

0.08 

0.81 

1.43 

2.94 

2.96 

1.5 

0.04 

1.43 

2.49 

4.55 

4.73 

0.05 

1.27 

2.02 

3.82 

3.89 

0.08 

1.11 

1.56 

3.13 

3.16 

0.09 

0.86 

1.43 

2.86 

2.88 

2.0 

0.05 

1.34 

1.84 

3.87 

3.92 

0.06 

1.34 

1.79 

3.35 

3.39 

0.07 

0.97 

1.50 

2.97 

2.98 

0.09 

0.78 

1.35 

2.75 

2.77 

3.0 

0.07 

1.50 

1.61 

3.47 

3.49 

0.06 

0.99 

1.58 

3.19 

3.22 

0.08 

0.98 

1.44 

2.85 

2.87 

0.09 

0.82 

1.38 

2.64 

2.65 

5.0 

0.06 

1.15 

1.52 

2.81 

2.84 

0.08 

1.11 

1.44 

2.75 

2.77 

0.09 

0.99 

1.40 

2.64 

2.65 

0.11 

0.78 

1.31 

2.52 

2.53 

10.0 

0.09 

1.06 

1.37 

2.36 

2.37 

0.09 

1.04 

1.51 

2.28 

2.30 

0.11 

0.80 

1.21 

2.24 

2.25 

0.12 

0.67 

1.22 

2.24 

2.24 

15.0 

0.09 

0.85 

1.21 

1.95 

1.96 

0.10 

0.81 

1.25 

1.95 

1.96 

0.12 

0.78 

1.16 

2.03 

2.04 

0.14 

0.60 

1.09 

2.00 

2.00 

20.0 

0.11 

0.86 

1.17 

1.81 

1.82 

0.11 

0.77 

1.12 

1.78 

1.80 

0.12 

0.65 

1.10 

1.83 

1.83 

0.15 

0.59 

1.05 

1.88 

1.88 

25.0 

0.12 

0.79 

1.10 

1.67 

1.69 

0.13 

0.74 

1.07 

1.68 

1.69 

0.14 

0.62 

1.03 

1.70 

1.71 

0.15 

0.53 

1.01 

1.73 

1.74 

30.0 

0.12 

0.67 

1.01 

1.54 

1.55 

0.14 

0.68 

1.00 

1.66 

1.67 

0.14 

0.56 

0.98 

1.60 

1.61 

0.16 

0.52 

0.99 

1.61 

1.61 

35.0 

0.13 

0.64 

0.97 

1.46 

1.46 

0.12 

0.55 

0.92 

1.40 

1.41 

0.15 

0.55 

0.96 

1.54 

1.55 

0.17 

0.49 

0.98 

1.60 

1.60 

40.0 

0.14 

0.61 

0.93 

1.43 

1.44 

0.13 

0.55 

0.90 

1.35 

1.35 

0.16 

0.52 

0.94 

1.48 

1.48 

0.18 

0.50 

0.96 

1.52 

1.52 

50.0 

0.13 

0.48 

0.85 

1.23 

1.23 

0.15 

0.49 

0.88 

1.34 

1.34 

0.15 

0.45 

0.91 

1.35 

1.35 

0.17 

0.42 

0.94 

1.39 

1.39 



Table 17: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.5, p = 50, ct = 10 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.02 

0.78 

1.58 

7.47 

7.60 

0.02 

0.56 

1.90 

4.49 

4.58 

0.03 

0.59 

1.41 

2.93 

2.98 

0.04 

0.41 

1.28 

2.56 

2.58 

0.1 

0.01 

0.62 

2.23 

8.60 

8.95 

0.02 

0.64 

1.58 

4.53 

4.64 

0.03 

0.52 

1.53 

2.97 

3.04 

0.04 

0.40 

1.25 

2.63 

2.65 

0.2 

0.01 

0.52 

1.97 

7.00 

7.40 

0.02 

0.57 

1.78 

4.30 

4.39 

0.03 

0.53 

1.43 

3.00 

3.05 

0.04 

0.41 

1.24 

2.59 

2.61 

0.3 

0.02 

0.47 

1.63 

5.90 

6.15 

0.02 

0.64 

1.55 

4.03 

4.12 

0.03 

0.41 

1.37 

2.73 

2.79 

0.05 

0.39 

1.21 

2.61 

2.63 

0.4 

0.02 

0.55 

1.82 

5.81 

6.04 

0.02 

0.51 

1.42 

3.84 

3.91 

0.03 

0.46 

1.41 

2.74 

2.78 

0.04 

0.36 

1.24 

2.53 

2.55 

0.5 

0.02 

0.62 

1.59 

5.31 

5.47 

0.02 

0.56 

1.53 

3.45 

3.50 

0.03 

0.48 

1.37 

2.80 

2.83 

0.05 

0.40 

1.22 

2.54 

2.55 

0.6 

0.01 

0.37 

1.63 

4.73 

4.85 

0.02 

0.63 

1.54 

3.71 

3.78 

0.03 

0.46 

1.39 

2.77 

2.80 

0.05 

0.37 

1.20 

2.57 

2.59 

0.7 

0.02 

0.59 

1.72 

5.20 

5.33 

0.02 

0.54 

1.62 

3.35 

3.41 

0.03 

0.45 

1.36 

2.78 

2.82 

0.05 

0.36 

1.22 

2.55 

2.56 

0.8 

0.02 

0.55 

1.46 

4.67 

4.76 

0.02 

0.53 

1.86 

3.34 

3.45 

0.04 

0.49 

1.39 

2.74 

2.76 

0.05 

0.37 

1.17 

2.51 

2.53 

0.9 

0.02 

0.54 

1.57 

4.12 

4.21 

0.03 

0.63 

1.49 

2.98 

3.04 

0.03 

0.50 

1.44 

2.65 

2.69 

0.05 

0.37 

1.19 

2.42 

2.44 

1.0 

0.02 

0.56 

1.73 

4.25 

4.33 

0.03 

0.58 

1.53 

3.31 

3.37 

0.04 

0.46 

1.33 

2.58 

2.60 

0.05 

0.34 

1.16 

2.36 

2.38 

1.5 

0.02 

0.56 

1.59 

3.54 

3.63 

0.02 

0.56 

1.61 

2.92 

3.02 

0.04 

0.45 

1.28 

2.59 

2.62 

0.06 

0.37 

1.14 

2.37 

2.38 

2.0 

0.02 

0.63 

1.52 

3.28 

3.36 

0.03 

0.57 

1.43 

2.91 

2.96 

0.04 

0.46 

1.31 

2.49 

2.51 

0.05 

0.36 

1.15 

2.36 

2.37 

3.0 

0.02 

0.56 

1.49 

2.52 

2.56 

0.03 

0.51 

1.45 

2.44 

2.47 

0.04 

0.44 

1.20 

2.35 

2.37 

0.06 

0.34 

1.11 

2.22 

2.23 

5.0 

0.03 

0.55 

1.30 

2.04 

2.08 

0.04 

0.55 

1.19 

2.14 

2.17 

0.05 

0.43 

1.16 

2.10 

2.11 

0.06 

0.31 

1.07 

2.02 

2.02 

10.0 

0.04 

0.41 

1.04 

1.53 

1.56 

0.05 

0.46 

1.02 

1.58 

1.60 

0.06 

0.35 

1.02 

1.73 

1.73 

0.07 

0.27 

1.01 

1.67 

1.67 

15.0 

0.04 

0.37 

0.93 

1.32 

1.34 

0.05 

0.35 

0.93 

1.34 

1.36 

0.05 

0.27 

0.96 

1.40 

1.41 

0.07 

0.24 

0.99 

1.54 

1.54 

20.0 

0.05 

0.32 

0.84 

1.17 

1.18 

0.05 

0.29 

0.84 

1.12 

1.12 

0.06 

0.26 

0.95 

1.35 

1.35 

0.07 

0.21 

0.98 

1.38 

1.38 

25.0 

0.05 

0.24 

0.79 

1.00 

1.00 

0.05 

0.24 

0.83 

1.04 

1.05 

0.06 

0.21 

0.93 

1.17 

1.17 

0.07 

0.20 

0.98 

1.29 

1.29 

30.0 

0.05 

0.21 

0.74 

0.88 

0.88 

0.05 

0.21 

0.82 

0.99 

0.99 

0.06 

0.20 

0.93 

1.12 

1.12 

0.07 

0.18 

0.98 

1.21 

1.21 

35.0 

0.05 

0.19 

0.75 

0.83 

0.83 

0.05 

0.20 

0.79 

0.88 

0.88 

0.06 

0.18 

0.94 

1.07 

1.07 

0.07 

0.18 

0.99 

1.19 

1.19 

40.0 

0.05 

0.19 

0.77 

0.81 

0.81 

0.05 

0.17 

0.85 

0.85 

0.85 

0.06 

0.17 

0.96 

1.04 

1.04 

0.07 

0.16 

0.99 

1.12 

1.12 

50.0 

0.05 

0.15 

0.82 

0.71 

0.71 

0.05 

0.15 

0.87 

0.78 

0.78 

0.06 

0.15 

0.99 

0.92 

0.92 

0.07 

0.15 

1.00 

1.08 

1.08 



Table 18: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.5, p = 50, a = 20 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.02 

0.60 

2.02 

8.19 

8.76 

0.01 

0.48 

1.63 

5.84 

6.04 

0.02 

0.47 

1.86 

5.08 

5.32 

0.02 

0.56 

1.52 

4.02 

4.13 

0.1 

0.01 

0.52 

1.65 

8.29 

8.73 

0.02 

0.56 

1.83 

6.45 

6.66 

0.02 

0.50 

1.56 

4.51 

4.60 

0.02 

0.46 

1.61 

4.36 

4.44 

0.2 

0.02 

0.63 

2.02 

7.89 

8.46 

0.02 

0.59 

1.70 

5.92 

6.06 

0.02 

0.54 

1.80 

4.40 

4.51 

0.02 

0.49 

1.81 

4.15 

4.27 

0.3 

0.02 

0.56 

1.99 

8.17 

8.57 

0.02 

0.70 

1.64 

6.43 

6.55 

0.02 

0.60 

1.68 

4.75 

4.90 

0.02 

0.56 

1.75 

4.11 

4.20 

0.4 

0.01 

0.49 

2.23 

9.43 

9.79 

0.02 

0.60 

1.78 

6.45 

6.58 

0.02 

0.57 

1.57 

4.13 

4.22 

0.02 

0.51 

1.69 

3.79 

3.88 

0.5 

0.01 

0.54 

2.04 

9.00 

9.40 

0.01 

0.58 

1.91 

6.46 

6.70 

0.02 

0.59 

1.91 

5.14 

5.29 

0.03 

0.53 

1.41 

3.95 

4.03 

0.6 

0.02 

0.50 

1.94 

7.58 

7.88 

0.02 

0.63 

1.90 

6.51 

6.73 

0.02 

0.55 

1.92 

4.62 

4.75 

0.02 

0.51 

1.63 

4.06 

4.17 

0.7 

0.02 

0.63 

2.37 

8.67 

9.10 

0.02 

0.58 

1.64 

5.60 

5.79 

0.02 

0.49 

1.66 

4.61 

4.74 

0.02 

0.49 

1.67 

4.22 

4.28 

0.8 

0.02 

0.66 

1.99 

7.54 

7.93 

0.02 

0.63 

1.88 

5.56 

5.70 

0.02 

0.63 

1.65 

4.55 

4.71 

0.03 

0.48 

1.74 

4.27 

4.42 

0.9 

0.02 

0.72 

2.04 

6.24 

6.38 

0.02 

0.58 

1.71 

5.92 

6.09 

0.02 

0.59 

1.72 

4.48 

4.64 

0.02 

0.51 

1.77 

3.89 

4.02 

1.0 

0.02 

0.48 

1.82 

6.76 

7.07 

0.02 

0.56 

1.98 

5.46 

5.62 

0.02 

0.63 

1.76 

4.58 

4.75 

0.02 

0.49 

1.62 

3.97 

4.07 

1.5 

0.02 

0.53 

1.90 

6.07 

6.26 

0.02 

0.58 

1.65 

5.04 

5.20 

0.02 

0.53 

1.42 

3.86 

3.96 

0.03 

0.53 

1.43 

4.07 

4.15 

2.0 

0.01 

0.52 

2.04 

5.73 

5.97 

0.02 

0.53 

1.84 

5.42 

5.59 

0.03 

0.56 

1.46 

3.95 

4.03 

0.03 

0.52 

1.58 

3.71 

3.79 

3.0 

0.02 

0.65 

1.69 

4.78 

4.97 

0.02 

0.56 

1.57 

4.36 

4.49 

0.02 

0.57 

1.79 

3.92 

3.99 

0.03 

0.46 

1.46 

3.63 

3.71 

5.0 

0.02 

0.48 

1.97 

3.96 

4.06 

0.02 

0.59 

1.60 

3.70 

3.75 

0.03 

0.50 

1.43 

3.53 

3.59 

0.03 

0.57 

1.67 

3.66 

3.71 

10.0 

0.02 

0.44 

1.41 

2.53 

2.58 

0.03 

0.54 

1.39 

2.84 

2.88 

0.03 

0.51 

1.40 

2.84 

2.89 

0.03 

0.49 

1.39 

2.83 

2.87 

15.0 

0.03 

0.55 

1.37 

2.48 

2.52 

0.03 

0.56 

1.42 

2.36 

2.42 

0.03 

0.44 

1.26 

2.39 

2.43 

0.04 

0.46 

1.27 

2.58 

2.61 

20.0 

0.03 

0.55 

1.28 

2.16 

2.21 

0.03 

0.49 

1.22 

2.04 

2.08 

0.04 

0.48 

1.25 

2.23 

2.25 

0.04 

0.39 

1.18 

2.16 

2.18 

25.0 

0.03 

0.52 

1.23 

2.05 

2.09 

0.03 

0.54 

1.19 

1.88 

1.91 

0.04 

0.42 

1.19 

2.04 

2.06 

0.04 

0.36 

1.14 

1.99 

2.01 

30.0 

0.04 

0.49 

1.13 

1.82 

1.86 

0.04 

0.48 

1.17 

1.82 

1.85 

0.04 

0.43 

1.15 

1.87 

1.89 

0.04 

0.39 

1.13 

1.99 

2.01 

35.0 

0.03 

0.46 

1.09 

1.60 

1.63 

0.04 

0.45 

1.08 

1.72 

1.74 

0.05 

0.44 

1.07 

1.85 

1.87 

0.05 

0.35 

1.07 

1.82 

1.83 

40.0 

0.04 

0.40 

1.07 

1.57 

1.60 

0.04 

0.44 

1.09 

1.59 

1.61 

0.04 

0.36 

1.03 

1.65 

1.67 

0.05 

0.34 

1.03 

1.72 

1.73 

50.0 

0.04 

0.42 

0.97 

1.37 

1.39 

0.05 

0.38 

0.99 

1.49 

1.51 

0.05 

0.36 

0.98 

1.53 

1.55 

0.05 

0.28 

0.97 

1.52 

1.52 



Table 19: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.9, p = 10, a = 10 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.09 

1.70 

1.73 

3.09 

3.09 

0.10 

1.47 

1.86 

2.89 

2.90 

0.13 

0.46 

1.14 

1.97 

1.97 

0.18 

0.32 

1.01 

1.56 

1.56 

0.1 

0.09 

1.52 

1.83 

3.13 

3.14 

0.09 

1.04 

1.59 

2.65 

2.65 

0.15 

0.45 

1.09 

1.93 

1.93 

0.19 

0.32 

1.00 

1.51 

1.51 

0.2 

0.10 

1.54 

1.58 

2.86 

2.87 

0.10 

0.92 

1.59 

2.51 

2.51 

0.14 

0.43 

1.08 

1.94 

1.94 

0.19 

0.32 

1.01 

1.49 

1.49 

0.3 

0.08 

1.60 

2.28 

3.27 

3.29 

0.10 

0.99 

1.68 

2.59 

2.60 

0.15 

0.40 

1.10 

1.86 

1.86 

0.20 

0.32 

1.00 

1.46 

1.46 

0.4 

0.10 

1.66 

1.57 

2.88 

2.89 

0.11 

0.93 

1.53 

2.54 

2.55 

0.15 

0.42 

1.12 

1.89 

1.89 

0.17 

0.28 

1.00 

1.47 

1.47 

0.5 

0.10 

1.44 

1.90 

3.06 

3.07 

0.10 

0.90 

1.51 

2.51 

2.51 

0.14 

0.38 

1.08 

1.82 

1.82 

0.20 

0.32 

1.00 

1.45 

1.45 

0.6 

0.09 

1.77 

2.09 

3.25 

3.26 

0.11 

0.74 

1.34 

2.30 

2.30 

0.13 

0.34 

1.07 

1.82 

1.83 

0.18 

0.28 

1.01 

1.46 

1.46 

0.7 

0.10 

1.40 

1.98 

2.99 

3.00 

0.12 

0.82 

1.32 

2.37 

2.38 

0.15 

0.37 

1.06 

1.80 

1.80 

0.20 

0.32 

1.00 

1.46 

1.46 

0.8 

0.09 

1.33 

1.81 

3.01 

3.02 

0.10 

0.77 

1.43 

2.47 

2.48 

0.15 

0.37 

1.06 

1.79 

1.79 

0.19 

0.31 

1.00 

1.44 

1.44 

0.9 

0.11 

1.43 

1.51 

2.67 

2.67 

0.11 

0.78 

1.36 

2.34 

2.34 

0.14 

0.34 

1.06 

1.76 

1.76 

0.20 

0.30 

1.00 

1.41 

1.41 

1.0 

0.09 

1.48 

1.99 

2.96 

2.96 

0.11 

0.85 

1.62 

2.41 

2.41 

0.16 

0.38 

1.06 

1.75 

1.75 

0.19 

0.29 

1.00 

1.43 

1.43 

1.5 

0.12 

1.08 

1.37 

2.43 

2.44 

0.11 

0.79 

1.57 

2.35 

2.36 

0.16 

0.35 

1.04 

1.70 

1.70 

0.20 

0.30 

1.00 

1.40 

1.40 

2.0 

0.09 

1.01 

1.72 

2.69 

2.70 

0.11 

0.50 

1.24 

2.10 

2.11 

0.17 

0.35 

1.03 

1.64 

1.64 

0.19 

0.28 

1.00 

1.38 

1.38 

3.0 

0.14 

0.92 

1.37 

2.26 

2.27 

0.15 

0.60 

1.19 

2.00 

2.00 

0.17 

0.33 

1.01 

1.57 

1.57 

0.22 

0.31 

1.00 

1.34 

1.34 

5.0 

0.14 

0.71 

1.27 

2.05 

2.05 

0.16 

0.48 

1.09 

1.81 

1.81 

0.18 

0.31 

1.00 

1.49 

1.49 

0.23 

0.32 

1.00 

1.31 

1.31 

10.0 

0.15 

0.46 

1.07 

1.67 

1.67 

0.18 

0.39 

1.01 

1.54 

1.54 

0.21 

0.32 

1.00 

1.34 

1.34 

0.26 

0.33 

1.00 

1.23 

1.23 

15.0 

0.18 

0.39 

1.00 

1.52 

1.52 

0.20 

0.36 

1.00 

1.40 

1.40 

0.24 

0.34 

1.00 

1.27 

1.27 

0.27 

0.33 

1.00 

1.19 

1.19 

20.0 

0.19 

0.37 

0.99 

1.39 

1.39 

0.22 

0.34 

0.99 

1.32 

1.32 

0.26 

0.34 

1.00 

1.22 

1.22 

0.28 

0.33 

1.00 

1.16 

1.16 

25.0 

0.21 

0.36 

0.99 

1.31 

1.31 

0.23 

0.33 

1.00 

1.26 

1.26 

0.27 

0.34 

1.00 

1.19 

1.19 

0.29 

0.34 

1.00 

1.14 

1.14 

30.0 

0.22 

0.35 

0.99 

1.25 

1.25 

0.25 

0.36 

1.00 

1.24 

1.24 

0.27 

0.33 

1.00 

1.17 

1.17 

0.30 

0.35 

1.00 

1.13 

1.13 

35.0 

0.25 

0.37 

0.99 

1.23 

1.23 

0.25 

0.35 

1.00 

1.19 

1.19 

0.26 

0.32 

1.00 

1.15 

1.15 

0.34 

0.39 

1.00 

1.11 

1.11 

40.0 

0.25 

0.35 

1.00 

1.18 

1.18 

0.25 

0.33 

1.00 

1.17 

1.17 

0.30 

0.36 

1.00 

1.14 

1.14 

0.31 

0.35 

1.00 

1.10 

1.10 

50.0 

0.26 

0.35 

1.00 

1.15 

1.15 

0.26 

0.33 

1.00 

1.13 

1.13 

0.30 

0.35 

1.00 

1.11 

1.11 

0.31 

0.35 

1.00 

1.09 

1.09 



Table 20: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.9, p = 10, a = 20 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.09 

1.88 

2.14 

3.27 

3.29 

0.09 

1.68 

1.74 

3.00 

3.01 

0.10 

0.99 

1.68 

2.77 

2.78 

0.11 

0.54 

1.30 

2.23 

2.24 

0.1 

0.09 

2.43 

2.23 

3.16 

3.17 

0.10 

1.44 

1.60 

2.85 

2.86 

0.10 

0.93 

1.54 

2.62 

2.63 

0.12 

0.55 

1.26 

2.23 

2.23 

0.2 

0.10 

1.50 

1.83 

2.98 

3.00 

0.08 

1.28 

1.65 

2.92 

2.92 

0.11 

0.91 

1.47 

2.61 

2.61 

0.13 

0.59 

1.24 

2.29 

2.29 

0.3 

0.10 

1.81 

1.93 

3.17 

3.18 

0.10 

1.59 

1.84 

3.08 

3.09 

0.11 

1.00 

1.50 

2.58 

2.59 

0.13 

0.49 

1.17 

2.12 

2.12 

0.4 

0.10 

1.85 

2.26 

3.31 

3.32 

0.08 

1.34 

1.92 

3.16 

3.17 

0.10 

0.89 

1.55 

2.54 

2.54 

0.11 

0.51 

1.28 

2.16 

2.16 

0.5 

0.10 

1.57 

1.78 

3.19 

3.19 

0.10 

1.37 

1.57 

2.88 

2.90 

0.10 

0.74 

1.43 

2.46 

2.46 

0.13 

0.54 

1.20 

2.16 

2.16 

0.6 

0.09 

1.70 

1.66 

2.91 

2.91 

0.09 

1.50 

2.06 

2.98 

2.99 

0.11 

0.89 

1.70 

2.67 

2.67 

0.13 

0.55 

1.21 

2.17 

2.17 

0.7 

0.10 

1.74 

1.99 

2.89 

2.90 

0.10 

1.25 

1.64 

2.84 

2.85 

0.13 

0.82 

1.32 

2.42 

2.42 

0.12 

0.49 

1.21 

2.16 

2.17 

0.8 

0.08 

1.48 

2.23 

3.39 

3.40 

0.08 

1.87 

2.70 

3.42 

3.44 

0.11 

0.74 

1.31 

2.51 

2.52 

0.11 

0.45 

1.20 

2.25 

2.26 

0.9 

0.08 

1.92 

2.32 

3.33 

3.34 

0.08 

1.36 

1.99 

3.07 

3.09 

0.10 

0.77 

1.65 

2.68 

2.68 

0.14 

0.51 

1.19 

2.17 

2.17 

1.0 

0.10 

1.53 

1.65 

2.92 

2.93 

0.09 

1.38 

1.82 

3.04 

3.06 

0.10 

0.77 

1.58 

2.58 

2.58 

0.13 

0.51 

1.23 

2.22 

2.22 

1.5 

0.09 

1.66 

2.08 

3.08 

3.09 

0.10 

1.46 

2.00 

2.93 

2.94 

0.10 

0.71 

1.43 

2.49 

2.50 

0.13 

0.44 

1.15 

2.08 

2.08 

2.0 

0.10 

1.56 

1.85 

3.01 

3.02 

0.10 

1.16 

1.50 

2.64 

2.64 

0.11 

0.76 

1.38 

2.38 

2.39 

0.13 

0.42 

1.13 

2.04 

2.04 

3.0 

0.09 

1.30 

2.16 

3.05 

3.06 

0.12 

1.03 

1.48 

2.53 

2.54 

0.13 

0.64 

1.25 

2.22 

2.22 

0.15 

0.44 

1.09 

1.96 

1.96 

5.0 

0.10 

1.12 

1.66 

2.77 

2.78 

0.12 

0.89 

1.48 

2.51 

2.51 

0.12 

0.56 

1.29 

2.19 

2.19 

0.16 

0.43 

1.06 

1.92 

1.92 

10.0 

0.10 

0.95 

1.66 

2.53 

2.53 

0.12 

0.77 

1.37 

2.25 

2.25 

0.13 

0.50 

1.18 

2.00 

2.00 

0.17 

0.38 

1.05 

1.71 

1.71 

15.0 

0.13 

0.79 

1.35 

2.17 

2.18 

0.13 

0.63 

1.23 

2.04 

2.04 

0.15 

0.42 

1.07 

1.82 

1.82 

0.15 

0.32 

1.03 

1.67 

1.67 

20.0 

0.14 

0.68 

1.23 

2.06 

2.06 

0.13 

0.51 

1.15 

1.87 

1.87 

0.15 

0.39 

1.08 

1.75 

1.75 

0.16 

0.32 

1.01 

1.57 

1.57 

25.0 

0.13 

0.57 

1.23 

1.87 

1.87 

0.14 

0.52 

1.13 

1.91 

1.91 

0.18 

0.41 

1.03 

1.62 

1.62 

0.18 

0.32 

1.01 

1.54 

1.54 

30.0 

0.16 

0.63 

1.13 

1.83 

1.83 

0.16 

0.49 

1.09 

1.74 

1.74 

0.17 

0.38 

1.06 

1.62 

1.62 

0.19 

0.32 

1.01 

1.48 

1.48 

35.0 

0.16 

0.49 

1.07 

1.76 

1.76 

0.17 

0.47 

1.06 

1.64 

1.64 

0.18 

0.35 

1.01 

1.54 

1.54 

0.20 

0.33 

1.00 

1.43 

1.43 

40.0 

0.16 

0.46 

1.06 

1.66 

1.66 

0.18 

0.42 

1.03 

1.61 

1.61 

0.19 

0.35 

1.01 

1.47 

1.47 

0.21 

0.33 

1.00 

1.41 

1.41 

50.0 

0.19 

0.47 

1.01 

1.56 

1.56 

0.17 

0.37 

1.04 

1.53 

1.53 

0.19 

0.33 

1.00 

1.41 

1.41 

0.21 

0.32 

1.00 

1.35 

1.35 


Table 21: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.9, p = 20, ct = 10 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.05 

1.26 

1.74 

4.79 

4.82 

0.06 

1.17 

1.55 

3.86 

3.88 

0.06 

0.44 

1.33 

3.08 

3.09 

0.09 

0.28 

1.04 

2.17 

2.18 

0.1 

0.05 

1.37 

2.08 

4.99 

5.02 

0.05 

1.19 

1.89 

4.43 

4.47 

0.07 

0.47 

1.33 

3.02 

3.03 

0.10 

0.27 

1.03 

2.04 

2.04 

0.2 

0.05 

1.40 

1.80 

4.74 

4.79 

0.05 

0.95 

1.67 

4.24 

4.27 

0.07 

0.41 

1.29 

2.82 

2.83 

0.09 

0.25 

1.03 

1.99 

1.99 

0.3 

0.04 

0.93 

1.66 

4.55 

4.59 

0.06 

0.85 

1.49 

3.69 

3.72 

0.07 

0.39 

1.21 

2.71 

2.71 

0.09 

0.23 

1.03 

1.98 

1.98 

0.4 

0.04 

1.13 

2.37 

5.19 

5.26 

0.04 

0.80 

1.70 

4.06 

4.10 

0.07 

0.38 

1.22 

2.67 

2.67 

0.10 

0.24 

1.01 

2.01 

2.01 

0.5 

0.06 

1.21 

1.53 

3.66 

3.68 

0.05 

0.93 

1.74 

3.80 

3.82 

0.08 

0.36 

1.17 

2.58 

2.59 

0.10 

0.24 

1.02 

1.96 

1.96 

0.6 

0.04 

1.40 

2.29 

4.94 

4.98 

0.05 

0.83 

1.63 

3.86 

3.87 

0.08 

0.40 

1.14 

2.60 

2.60 

0.10 

0.25 

1.02 

1.93 

1.93 

0.7 

0.05 

1.20 

1.71 

4.60 

4.63 

0.06 

0.74 

1.39 

3.30 

3.32 

0.07 

0.34 

1.13 

2.54 

2.54 

0.09 

0.23 

1.02 

1.92 

1.92 

0.8 

0.06 

1.08 

1.62 

3.92 

3.95 

0.05 

0.78 

1.55 

3.49 

3.51 

0.07 

0.36 

1.17 

2.64 

2.65 

0.09 

0.23 

1.01 

1.94 

1.94 

0.9 

0.05 

1.35 

1.95 

4.41 

4.47 

0.05 

0.79 

1.69 

3.66 

3.71 

0.07 

0.35 

1.17 

2.64 

2.64 

0.09 

0.21 

1.02 

1.97 

1.97 

1.0 

0.05 

1.13 

1.95 

4.66 

4.68 

0.06 

0.81 

1.69 

3.78 

3.81 

0.08 

0.35 

1.16 

2.50 

2.50 

0.10 

0.24 

1.02 

1.90 

1.90 

1.5 

0.05 

1.07 

1.75 

4.05 

4.10 

0.05 

0.70 

1.52 

3.53 

3.56 

0.08 

0.31 

1.10 

2.31 

2.31 

0.09 

0.20 

1.01 

1.86 

1.86 

2.0 

0.05 

1.12 

1.90 

4.06 

4.08 

0.06 

0.64 

1.50 

3.35 

3.37 

0.08 

0.30 

1.08 

2.29 

2.29 

0.11 

0.22 

1.01 

1.76 

1.76 

3.0 

0.06 

0.87 

1.51 

3.34 

3.36 

0.06 

0.53 

1.32 

2.79 

2.80 

0.08 

0.28 

1.07 

2.20 

2.20 

0.10 

0.21 

1.00 

1.71 

1.71 

5.0 

0.06 

0.68 

1.49 

2.78 

2.80 

0.08 

0.46 

1.23 

2.51 

2.52 

0.09 

0.24 

1.04 

1.97 

1.97 

0.11 

0.21 

1.00 

1.64 

1.64 

10.0 

0.08 

0.48 

1.15 

2.18 

2.19 

0.09 

0.33 

1.11 

2.02 

2.02 

0.11 

0.24 

1.01 

1.70 

1.70 

0.12 

0.21 

1.00 

1.48 

1.48 

15.0 

0.09 

0.39 

1.09 

1.86 

1.86 

0.10 

0.29 

1.02 

1.73 

1.73 

0.12 

0.24 

1.00 

1.56 

1.56 

0.14 

0.21 

1.00 

1.37 

1.37 

20.0 

0.10 

0.31 

1.02 

1.70 

1.70 

0.11 

0.28 

1.01 

1.62 

1.62 

0.12 

0.21 

1.00 

1.44 

1.44 

0.14 

0.20 

1.00 

1.33 

1.33 

25.0 

0.11 

0.30 

1.00 

1.59 

1.59 

0.12 

0.27 

1.00 

1.52 

1.52 

0.14 

0.24 

1.00 

1.39 

1.39 

0.15 

0.21 

1.00 

1.28 

1.28 

30.0 

0.11 

0.27 

1.00 

1.48 

1.48 

0.13 

0.25 

1.00 

1.45 

1.45 

0.14 

0.22 

1.00 

1.33 

1.33 

0.15 

0.21 

1.00 

1.25 

1.25 

35.0 

0.11 

0.25 

0.99 

1.38 

1.38 

0.13 

0.25 

1.00 

1.37 

1.37 

0.14 

0.22 

1.00 

1.29 

1.29 

0.17 

0.22 

1.00 

1.23 

1.23 

40.0 

0.13 

0.27 

1.00 

1.38 

1.38 

0.13 

0.23 

1.00 

1.33 

1.33 

0.14 

0.21 

1.00 

1.25 

1.25 

0.15 

0.20 

1.00 

1.21 

1.21 

50.0 

0.14 

0.25 

1.00 

1.29 

1.29 

0.13 

0.21 

1.00 

1.26 

1.26 

0.15 

0.21 

1.00 

1.22 

1.22 

0.17 

0.21 


1.17 

1.17 



Table 22: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.9, p = 20, a = 20 
for different values of k. 



LSE 

IPT 

II 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.04 

1.51 

2.11 

5.76 

5.81 

0.04 

1.12 

1.68 

4.39 

4.42 

0.05 

1.03 

1.80 

4.46 

4.48 

0.05 

0.63 

1.63 

3.69 

3.72 

0.1 

0.04 

1.66 

2.35 

5.11 

5.13 

0.05 

1.16 

1.43 

4.26 

4.30 

0.04 

1.01 

2.02 

4.57 

4.62 

0.05 

0.51 

1.44 

3.78 

3.79 

0.2 

0.03 

1.12 

2.21 

5.48 

5.71 

0.06 

1.37 

1.65 

4.51 

4.55 

0.05 

0.77 

1.65 

3.91 

3.96 

0.06 

0.61 

1.39 

3.56 

3.58 

0.3 

0.04 

1.21 

1.87 

5.11 

5.14 

0.05 

1.22 

1.96 

5.13 

5.17 

0.06 

1.05 

1.94 

4.48 

4.52 

0.07 

0.59 

1.38 

3.34 

3.35 

0.4 

0.05 

1.33 

1.97 

5.60 

5.65 

0.05 

1.19 

1.98 

4.86 

4.91 

0.06 

1.02 

1.88 

4.48 

4.50 

0.06 

0.54 

1.37 

3.51 

3.52 

0.5 

0.05 

1.23 

2.00 

5.29 

5.35 

0.04 

1.01 

2.06 

4.94 

4.97 

0.05 

0.84 

1.61 

3.81 

3.83 

0.07 

0.53 

1.33 

3.07 

3.08 

0.6 

0.05 

1.51 

2.39 

5.05 

5.10 

0.05 

1.35 

1.97 

4.40 

4.42 

0.05 

0.73 

1.47 

3.82 

3.84 

0.06 

0.49 

1.36 

3.48 

3.50 

0.7 

0.04 

1.29 

2.13 

5.05 

5.12 

0.06 

1.16 

1.53 

4.37 

4.41 

0.06 

1.04 

1.54 

3.76 

3.78 

0.05 

0.45 

1.50 

3.61 

3.64 

0.8 

0.04 

1.33 

2.31 

6.10 

6.17 

0.05 

1.15 

1.81 

4.70 

4.76 

0.05 

0.75 

1.71 

4.11 

4.15 

0.06 

0.45 

1.25 

3.17 

3.19 

0.9 

0.05 

1.46 

1.78 

4.60 

4.62 

0.05 

0.99 

1.66 

4.42 

4.44 

0.05 

0.75 

1.62 

3.76 

3.82 

0.07 

0.49 

1.33 

3.28 

3.29 

1.0 

0.05 

1.50 

2.06 

5.17 

5.24 

0.05 

1.31 

2.03 

4.86 

4.92 

0.05 

0.75 

1.75 

4.45 

4.50 

0.07 

0.51 

1.28 

3.20 

3.21 

1.5 

0.04 

1.86 

2.89 

5.82 

5.87 

0.05 

1.31 

1.76 

4.28 

4.30 

0.05 

0.83 

1.70 

3.91 

3.95 

0.07 

0.50 

1.23 

3.04 

3.05 

2.0 

0.05 

1.14 

1.77 

4.51 

4.57 

0.04 

0.84 

1.91 

4.59 

4.64 

0.06 

0.63 

1.35 

3.29 

3.31 

0.06 

0.38 

1.30 

3.22 

3.22 

3.0 

0.05 

1.32 

2.40 

5.03 

5.07 

0.06 

0.94 

1.55 

4.02 

4.03 

0.06 

0.66 

1.37 

3.44 

3.47 

0.06 

0.40 

1.29 

3.13 

3.14 

5.0 

0.05 

1.19 

1.93 

4.21 

4.24 

0.06 

0.98 

1.77 

3.99 

4.01 

0.06 

0.52 

1.42 

3.17 

3.18 

0.07 

0.39 

1.21 

2.90 

2.91 

10.0 

0.05 

0.88 

1.46 

3.58 

3.60 

0.06 

0.68 

1.40 

3.08 

3.10 

0.07 

0.46 

1.23 

2.87 

2.88 

0.08 

0.33 

1.16 

2.50 

2.50 

15.0 

0.05 

0.77 

1.48 

3.07 

3.09 

0.07 

0.65 

1.29 

2.88 

2.90 

0.08 

0.44 

1.20 

2.61 

2.61 

0.09 

0.32 

1.08 

2.26 

2.26 

20.0 

0.07 

0.75 

1.42 

2.96 

2.98 

0.07 

0.56 

1.35 

2.66 

2.67 

0.08 

0.39 

1.14 

2.44 

2.45 

0.10 

0.32 

1.06 

2.14 

2.14 

25.0 

0.06 

0.60 

1.33 

2.53 

2.53 

0.08 

0.56 

1.25 

2.53 

2.54 

0.07 

0.33 

1.13 

2.32 

2.33 

0.09 

0.27 

1.04 

2.08 

2.08 

30.0 

0.08 

0.58 

1.26 

2.38 

2.39 

0.09 

0.47 

1.14 

2.40 

2.40 

0.10 

0.35 

1.08 

2.12 

2.12 

0.10 

0.30 

1.04 

1.87 

1.87 

35.0 

0.08 

0.53 

1.23 

2.33 

2.33 

0.08 

0.43 

1.16 

2.26 

2.26 

0.09 

0.33 

1.08 

2.09 

2.09 

0.09 

0.25 

1.03 

1.91 

1.91 

40.0 

0.08 

0.48 

1.19 

2.16 

2.17 

0.08 

0.39 

1.12 

2.08 

2.08 

0.09 

0.29 

1.04 

1.92 

1.92 

0.10 

0.25 

1.02 

1.82 

1.82 

50.0 

0.09 

0.42 

1.13 

2.11 

2.11 

0.09 

0.33 

1.05 

2.02 

2.02 

0.10 

0.28 

1.03 

1.81 

1.81 

0.11 

0.25 

1.01 

1.70 

1.70 



Table 23: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.9, p = 50, ct = 10 
for different values of k. 



LSE 

IPT 

fc = 0 
PTLE 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.01 

0.50 

2.72 

12.93 

13.75 

0.02 

0.57 

1.93 

6.34 

6.62 

0.02 

0.29 

1.50 

4.44 

4.49 

0.03 

0.20 

1.16 

3.47 

3.48 

0.1 

0.02 

0.54 

2.03 

9.09 

9.46 

0.01 

0.42 

1.77 

6.33 

6.58 

0.02 

0.28 

1.39 

4.63 

4.68 

0.03 

0.17 

1.13 

3.26 

3.26 

0.2 

0.01 

0.55 

2.17 

7.98 

8.39 

0.02 

0.42 

1.84 

6.48 

6.80 

0.02 

0.28 

1.40 

4.62 

4.67 

0.02 

0.15 

1.12 

3.22 

3.22 

0.3 

0.01 

0.52 

1.94 

7.59 

7.84 

0.01 

0.53 

2.31 

7.46 

7.83 

0.02 

0.28 

1.46 

4.80 

4.87 

0.02 

0.14 

1.17 

3.37 

3.37 

0.4 

0.02 

0.49 

1.93 

5.46 

5.53 

0.02 

0.50 

2.08 

7.17 

7.41 

0.02 

0.25 

1.31 

3.98 

4.00 

0.03 

0.16 

1.14 

3.17 

3.17 

0.5 

0.01 

0.52 

2.05 

7.33 

7.66 

0.02 

0.38 

1.52 

5.92 

6.12 

0.02 

0.25 

1.51 

4.37 

4.41 

0.03 

0.15 

1.11 

3.17 

3.17 

0.6 

0.01 

0.46 

1.60 

6.39 

6.54 

0.01 

0.38 

1.68 

6.38 

6.67 

0.02 

0.29 

1.37 

4.17 

4.23 

0.03 

0.15 

1.10 

3.10 

3.11 

0.7 

0.01 

0.44 

1.77 

7.90 

8.25 

0.02 

0.51 

1.71 

6.27 

6.47 

0.02 

0.27 

1.28 

4.13 

4.15 

0.03 

0.15 

1.10 

3.09 

3.09 

0.8 

0.02 

0.55 

1.82 

7.81 

8.19 

0.02 

0.38 

1.52 

5.54 

5.70 

0.02 

0.26 

1.37 

4.23 

4.27 

0.03 

0.14 

1.12 

3.08 

3.08 

0.9 

0.01 

0.53 

2.11 

7.24 

7.42 

0.02 

0.44 

1.77 

6.20 

6.33 

0.02 

0.23 

1.34 

3.96 

4.00 

0.03 

0.15 

1.09 

3.12 

3.12 

1.0 

0.01 

0.49 

1.94 

7.08 

7.47 

0.02 

0.45 

1.55 

5.47 

5.58 

0.02 

0.25 

1.27 

3.95 

3.97 

0.03 

0.13 

1.14 

2.91 

2.91 

1.5 

0.02 

0.69 

1.62 

5.30 

5.40 

0.02 

0.40 

1.61 

4.97 

5.09 

0.02 

0.23 

1.25 

3.88 

3.90 

0.03 

0.13 

1.08 

2.96 

2.96 

2.0 

0.01 

0.46 

1.73 

5.59 

5.78 

0.02 

0.37 

1.52 

4.77 

4.85 

0.02 

0.24 

1.28 

3.70 

3.71 

0.03 

0.15 

1.07 

2.78 

2.78 

3.0 

0.02 

0.53 

1.67 

5.29 

5.38 

0.02 

0.30 

1.39 

4.15 

4.21 

0.03 

0.20 

1.16 

3.23 

3.24 

0.03 

0.12 

1.04 

2.54 

2.54 

5.0 

0.02 

0.53 

1.52 

4.09 

4.16 

0.02 

0.32 

1.40 

3.71 

3.74 

0.03 

0.18 

1.12 

2.93 

2.93 

0.03 

0.13 

1.03 

2.32 

2.32 

10.0 

0.03 

0.40 

1.32 

3.16 

3.18 

0.03 

0.26 

1.23 

2.80 

2.81 

0.03 

0.15 

1.06 

2.39 

2.39 

0.04 

0.11 

1.01 

2.05 

2.05 

15.0 

0.03 

0.29 

1.21 

2.59 

2.61 

0.03 

0.26 

1.10 

2.43 

2.44 

0.04 

0.14 

1.02 

2.14 

2.14 

0.04 

0.10 

1.00 

1.89 

1.89 

20.0 

0.03 

0.23 

1.09 

2.23 

2.25 

0.03 

0.18 

1.07 

2.15 

2.15 

0.03 

0.12 

1.01 

1.99 

1.99 

0.04 

0.10 

1.00 

1.71 

1.71 

25.0 

0.03 

0.20 

1.08 

2.04 

2.04 

0.03 

0.16 

1.03 

1.97 

1.97 

0.04 

0.12 

1.01 

1.78 

1.78 

0.04 

0.09 

1.00 

1.62 

1.62 

30.0 

0.03 

0.19 

1.06 

1.92 

1.92 

0.03 

0.15 

1.02 

1.81 

1.81 

0.04 

0.11 

1.00 

1.69 

1.69 

0.04 

0.09 

1.00 

1.56 

1.56 

35.0 

0.03 

0.17 

1.01 

1.75 

1.75 

0.04 

0.14 

1.01 

1.70 

1.70 

0.04 

0.11 

1.00 

1.59 

1.59 

0.04 

0.09 

1.00 

1.50 

1.50 

40.0 

0.04 

0.17 

1.00 

1.73 

1.73 

0.04 

0.15 

1.00 

1.67 

1.67 

0.04 

0.10 

1.00 

1.52 

1.52 

0.05 

0.10 

1.00 

1.45 

1.45 

50.0 

0.04 

0.14 

1.00 

1.52 

1.52 

0.04 

0.12 

1.00 

1.49 

1.49 

0.04 

0.10 

1.00 

1.45 

1.45 

0.06 

0.10 

1.00 

1.40 

1.40 



Table 24: Relative efficiencies of IPX, PTLE, SLE, and PSLE with respect to LASSO estimator when n = 100, r = 0.9, p = 50, a = 20 
for different values of k. 



LSE 

IPT 

fc = 0 
PTLE 

SLE 

PSLE 

LSE 

IPT 

k = 1 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 3 
PTLE 

SLE 

PSLE 

LSE 

IPT 

fc = 5 
PTLE 

SLE 

PSLE 

0.0 

0.01 

0.55 

2.34 

10.73 

11.28 

0.01 

0.53 

2.17 

9.25 

9.67 

0.02 

0.51 

1.99 

7.74 

7.99 

0.02 

0.37 

1.72 

6.29 

6.52 

0.1 

0.01 

0.45 

2.35 

9.83 

10.31 

0.02 

0.53 

1.61 

6.96 

7.12 

0.02 

0.46 

1.76 

7.18 

7.32 

0.02 

0.33 

1.52 

5.64 

5.75 

0.2 

0.01 

0.44 

1.92 

9.03 

9.34 

0.01 

0.53 

2.16 

9.44 

9.98 

0.02 

0.42 

1.71 

5.85 

6.03 

0.02 

0.34 

1.61 

5.61 

5.74 

0.3 

0.01 

0.49 

2.20 

9.35 

9.84 

0.01 

0.48 

1.87 

7.93 

8.36 

0.02 

0.46 

1.90 

6.61 

6.77 

0.02 

0.32 

1.62 

6.08 

6.19 

0.4 

0.01 

0.55 

2.00 

8.62 

8.94 

0.02 

0.50 

1.78 

7.63 

7.82 

0.02 

0.56 

1.84 

7.30 

7.49 

0.02 

0.39 

1.58 

5.74 

5.85 

0.5 

0.01 

0.47 

1.93 

8.90 

9.33 

0.01 

0.52 

2.06 

7.89 

8.11 

0.02 

0.43 

1.78 

6.74 

6.96 

0.02 

0.34 

1.63 

5.81 

5.92 

0.6 

0.01 

0.52 

2.00 

8.36 

8.67 

0.02 

0.51 

1.91 

7.50 

7.80 

0.02 

0.43 

1.78 

6.87 

7.12 

0.02 

0.32 

1.56 

5.79 

5.94 

0.7 

0.02 

0.55 

1.82 

7.84 

8.11 

0.02 

0.51 

1.70 

7.12 

7.35 

0.02 

0.45 

1.56 

6.18 

6.44 

0.02 

0.37 

1.58 

5.64 

5.76 

0.8 

0.02 

0.62 

1.83 

7.84 

8.07 

0.02 

0.56 

1.86 

7.96 

8.28 

0.02 

0.38 

1.90 

6.78 

6.99 

0.02 

0.36 

1.55 

5.75 

5.85 

0.9 

0.01 

0.56 

2.21 

8.91 

9.39 

0.01 

0.43 

1.72 

7.59 

8.01 

0.02 

0.50 

1.65 

6.22 

6.35 

0.02 

0.33 

1.49 

5.50 

5.60 

1.0 

0.01 

0.57 

1.95 

9.21 

9.52 

0.01 

0.49 

1.93 

7.20 

7.52 

0.02 

0.46 

1.90 

6.50 

6.71 

0.02 

0.31 

1.48 

5.10 

5.18 

1.5 

0.01 

0.55 

2.03 

8.42 

8.74 

0.01 

0.45 

1.86 

6.77 

6.97 

0.02 

0.43 

1.89 

6.65 

6.83 

0.02 

0.31 

1.56 

5.93 

6.07 

2.0 

0.02 

0.55 

1.68 

6.46 

6.65 

0.02 

0.52 

2.23 

7.95 

8.29 

0.02 

0.39 

1.55 

5.74 

5.90 

0.02 

0.32 

1.52 

5.21 

5.27 

3.0 

0.02 

0.58 

1.73 

6.72 

6.89 

0.01 

0.49 

1.87 

6.88 

7.12 

0.02 

0.39 

1.69 

6.18 

6.34 

0.02 

0.29 

1.55 

5.88 

5.99 

5.0 

0.02 

0.54 

1.71 

5.72 

5.87 

0.02 

0.44 

1.66 

5.32 

5.44 

0.02 

0.32 

1.65 

5.49 

5.61 

0.02 

0.27 

1.39 

4.60 

4.64 

10.0 

0.02 

0.45 

1.49 

5.05 

5.15 

0.02 

0.41 

1.45 

4.88 

4.95 

0.02 

0.33 

1.49 

4.60 

4.65 

0.02 

0.24 

1.36 

4.27 

4.31 

15.0 

0.02 

0.40 

1.46 

4.55 

4.67 

0.02 

0.42 

1.55 

4.67 

4.74 

0.02 

0.30 

1.36 

4.03 

4.07 

0.02 

0.19 

1.26 

3.69 

3.71 

20.0 

0.02 

0.43 

1.55 

4.09 

4.16 

0.02 

0.37 

1.51 

4.25 

4.31 

0.02 

0.28 

1.35 

3.94 

3.98 

0.03 

0.21 

1.19 

3.22 

3.23 

25.0 

0.02 

0.42 

1.38 

3.71 

3.75 

0.02 

0.32 

1.42 

3.53 

3.58 

0.02 

0.23 

1.28 

3.32 

3.35 

0.02 

0.17 

1.18 

3.19 

3.20 

30.0 

0.02 

0.36 

1.47 

3.60 

3.65 

0.02 

0.33 

1.33 

3.14 

3.17 

0.03 

0.24 

1.23 

3.14 

3.15 

0.03 

0.18 

1.14 

2.94 

2.95 

35.0 

0.02 

0.35 

1.36 

3.27 

3.33 

0.03 

0.31 

1.24 

3.09 

3.12 

0.03 

0.23 

1.17 

3.01 

3.02 

0.03 

0.16 

1.10 

2.74 

2.74 

40.0 

0.02 

0.32 

1.32 

2.99 

3.03 

0.02 

0.26 

1.21 

2.82 

2.83 

0.03 

0.22 

1.17 

2.89 

2.90 

0.03 

0.17 

1.10 

2.64 

2.64 

50.0 

0.02 

0.28 

1.25 

2.69 

2.72 

0.02 

0.23 

1.21 

2.60 

2.61 

0.03 

0.20 

1.13 

2.63 

2.64 

0.03 

0.15 

1.07 

2.45 

2.45 
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